2024-05-08T17:05:19.568 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T17:05:19.568 DEBUG:teuthology.run:Teuthology command: teuthology --description rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} --name lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi --verbose --archive /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758 --owner scheduled_lflores@teuthology -- /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/orig.config.yaml 2024-05-08T17:05:19.609 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T17:05:19.694 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758 branch: wip-lflores-testing-2-2024-05-07-1606-squid description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} email: null first_in_suite: false job_id: '7697758' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 10 - volumes: count: 4 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lflores-testing-2-2024-05-07-1606-squid ceph: conf: global: lockdep: true mon client directed command retry: 5 mon election default strategy: 3 ms bind msgr1: false ms bind msgr2: true ms inject socket failures: 5000 ms type: async mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 mon scrub interval: 300 paxos service trim min: 10 osd: bluefs allocator: hybrid bluestore allocator: hybrid bluestore block size: 96636764160 bluestore fsck on mount: true bluestore zero block detection: true debug bluefs: 20 debug bluestore: 20 debug monc: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd blocked scrub grace period: 3600 osd debug verify cached snaps: true osd debug verify missing on start: true osd failsafe full ratio: 0.95 osd max markdown count: 1000 osd mclock override recovery settings: true osd mclock profile: high_recovery_ops osd objectstore: bluestore osd op queue: debug_random osd op queue cut off: debug_random osd shutdown pgref assert: true flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - but it is still running - objects unfound and apparently lost - overall HEALTH_ - \(OSDMAP_FLAGS\) - \(OSD_ - \(PG_ - \(POOL_ - \(CACHE_POOL_ - \(SMALLER_PGP_NUM\) - \(OBJECT_ - \(SLOW_OPS\) - \(REQUEST_SLOW\) - \(TOO_FEW_PGS\) - slow request - timeout on replica - late reservation from - MON_DOWN - OSDMAP_FLAGS - OSD_DOWN - PG_DEGRADED - PG_AVAILABILITY - POOL_APP_NOT_ENABLED - \(OSD_SLOW_PING_TIME - overall HEALTH_ - \(MON_DOWN\) - \(OSDMAP_FLAGS\) - \(SMALLER_PGP_NUM\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW OPS\) - slow request sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lflores-testing-2-2024-05-07-1606-squid sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 owner: scheduled_lflores@teuthology priority: 75 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - node-exporter.a - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - prometheus.a - node-exporter.b seed: 8082 sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 sleep_before_teardown: 0 subset: 111/120000 suite: rados suite_branch: wip-lflores-testing-2-2024-05-07-1606-squid suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 targets: smithi154.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJrT+ufi9mx4WgjWMF+gLBS30zM9CR4KA5V8u5nsFkF+SR6QfsZY+x96tiHJsKRFWoSo1jbwYdHkLpl8OxhbPk= smithi183.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0wuaKY8ZS4W3t6jNFd7CeZ9saC5tZuFO8Ko86e3r1b5g9YwFdiC3zamVDAWWV3Y6ZAs8FYVONJpcCYg2e7Q6U= tasks: - install: null - ceph: null - thrashosds: chance_pgnum_grow: 1 chance_pgnum_shrink: 1 chance_pgpnum_fix: 1 timeout: 1200 - mon_recovery: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_14:59:36 tube: smithi user: lflores verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T17:05:19.695 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa; will attempt to use it 2024-05-08T17:05:19.695 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa/tasks 2024-05-08T17:05:19.696 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T17:05:19.697 INFO:teuthology.task.internal:Checking packages... 2024-05-08T17:05:19.722 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '95a4ad3c90f2f12f920e87725ad8d9002f948722' 2024-05-08T17:05:19.722 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T17:05:19.722 INFO:teuthology.packaging:ref: None 2024-05-08T17:05:19.723 INFO:teuthology.packaging:tag: None 2024-05-08T17:05:19.723 INFO:teuthology.packaging:branch: wip-lflores-testing-2-2024-05-07-1606-squid 2024-05-08T17:05:19.723 INFO:teuthology.packaging:sha1: 95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T17:05:19.723 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lflores-testing-2-2024-05-07-1606-squid 2024-05-08T17:05:19.947 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2517.g95a4ad3c 2024-05-08T17:05:19.948 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T17:05:19.955 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T17:05:19.955 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T17:05:19.962 INFO:teuthology.task.internal:Saving configuration 2024-05-08T17:05:19.977 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T17:05:19.983 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T17:05:20.007 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi154.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 17:00:02.156944', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJrT+ufi9mx4WgjWMF+gLBS30zM9CR4KA5V8u5nsFkF+SR6QfsZY+x96tiHJsKRFWoSo1jbwYdHkLpl8OxhbPk='} 2024-05-08T17:05:20.031 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi183.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 17:00:02.155728', 'locked_by': 'scheduled_lflores@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0wuaKY8ZS4W3t6jNFd7CeZ9saC5tZuFO8Ko86e3r1b5g9YwFdiC3zamVDAWWV3Y6ZAs8FYVONJpcCYg2e7Q6U='} 2024-05-08T17:05:20.031 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T17:05:20.037 INFO:teuthology.task.internal:roles: ubuntu@smithi154.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'] 2024-05-08T17:05:20.037 INFO:teuthology.task.internal:roles: ubuntu@smithi183.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b'] 2024-05-08T17:05:20.037 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T17:05:20.109 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc740373e50>, signals=[15]) 2024-05-08T17:05:20.110 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T17:05:20.115 INFO:teuthology.task.internal:Opening connections... 2024-05-08T17:05:20.116 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:05:20.117 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:05:20.192 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:05:20.193 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:05:20.267 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T17:05:20.274 DEBUG:teuthology.orchestra.run.smithi154:> uname -m 2024-05-08T17:05:20.293 INFO:teuthology.orchestra.run.smithi154.stdout:x86_64 2024-05-08T17:05:20.293 DEBUG:teuthology.orchestra.run.smithi154:> cat /etc/os-release 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:NAME="CentOS Stream" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION="9" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:ID="centos" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:ID_LIKE="rhel fedora" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:VERSION_ID="9" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:PLATFORM_ID="platform:el9" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:ANSI_COLOR="0;31" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:LOGO="fedora-logo-icon" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T17:05:20.351 INFO:teuthology.orchestra.run.smithi154.stdout:HOME_URL="https://centos.org/" 2024-05-08T17:05:20.352 INFO:teuthology.orchestra.run.smithi154.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T17:05:20.352 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T17:05:20.352 INFO:teuthology.orchestra.run.smithi154.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T17:05:20.352 INFO:teuthology.lock.ops:Updating smithi154.front.sepia.ceph.com on lock server 2024-05-08T17:05:20.378 DEBUG:teuthology.orchestra.run.smithi183:> uname -m 2024-05-08T17:05:20.395 INFO:teuthology.orchestra.run.smithi183.stdout:x86_64 2024-05-08T17:05:20.395 DEBUG:teuthology.orchestra.run.smithi183:> cat /etc/os-release 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:NAME="CentOS Stream" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION="9" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:ID="centos" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:ID_LIKE="rhel fedora" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:VERSION_ID="9" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:PLATFORM_ID="platform:el9" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:ANSI_COLOR="0;31" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:LOGO="fedora-logo-icon" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:HOME_URL="https://centos.org/" 2024-05-08T17:05:20.452 INFO:teuthology.orchestra.run.smithi183.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T17:05:20.453 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T17:05:20.453 INFO:teuthology.orchestra.run.smithi183.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T17:05:20.453 INFO:teuthology.lock.ops:Updating smithi183.front.sepia.ceph.com on lock server 2024-05-08T17:05:20.478 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T17:05:20.487 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T17:05:20.540 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T17:05:20.540 DEBUG:teuthology.orchestra.run.smithi154:> test '!' -e /home/ubuntu/cephtest 2024-05-08T17:05:20.542 DEBUG:teuthology.orchestra.run.smithi183:> test '!' -e /home/ubuntu/cephtest 2024-05-08T17:05:20.558 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T17:05:20.564 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T17:05:20.564 DEBUG:teuthology.orchestra.run.smithi154:> test -z $(ls -A /var/lib/ceph) 2024-05-08T17:05:20.599 DEBUG:teuthology.orchestra.run.smithi183:> test -z $(ls -A /var/lib/ceph) 2024-05-08T17:05:20.699 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T17:05:20.752 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T17:05:20.768 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-05-08T17:05:20.768 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mon.b': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-05-08T17:05:20.768 DEBUG:teuthology.orchestra.run.smithi154:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T17:05:20.769 DEBUG:teuthology.orchestra.run.smithi183:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T17:05:20.784 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:05:20.785 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-05-08T17:05:20.786 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:05:20.786 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-05-08T17:05:20.841 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-437.el9.x86_64 2024-05-08T17:05:20.842 INFO:teuthology.task.kernel:Running kernel on smithi183: 5.14.0-437.el9.x86_64 2024-05-08T17:05:20.842 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum install -y kernel 2024-05-08T17:05:20.844 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-437.el9.x86_64 2024-05-08T17:05:20.844 INFO:teuthology.task.kernel:Running kernel on smithi154: 5.14.0-437.el9.x86_64 2024-05-08T17:05:20.845 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum install -y kernel 2024-05-08T17:05:22.975 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 15 MB/s | 8.1 MB 00:00 2024-05-08T17:05:23.815 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 6.0 MB/s | 8.1 MB 00:01 2024-05-08T17:05:39.274 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 19 MB 00:15 2024-05-08T17:05:40.798 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:15 2024-05-08T17:05:44.807 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.2 MB 00:00 2024-05-08T17:05:46.290 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 1.8 MB/s | 6.2 MB 00:03 2024-05-08T17:05:46.415 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 62 kB/s | 16 kB 00:00 2024-05-08T17:05:47.433 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 24 MB/s | 21 MB 00:00 2024-05-08T17:05:47.999 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 46 kB/s | 16 kB 00:00 2024-05-08T17:05:49.776 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-08T17:05:52.508 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 22 kB/s | 1.7 kB 00:00 2024-05-08T17:05:54.498 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T17:05:54.498 INFO:teuthology.orchestra.run.smithi183.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T17:05:54.594 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:05:54.609 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:05:54.609 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-05-08T17:05:54.609 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:05:54.609 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:05:54.610 INFO:teuthology.orchestra.run.smithi183.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T17:05:54.610 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:05:54.610 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T17:05:54.610 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T17:05:54.610 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout:Install 4 Packages 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:05:54.611 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 78 M 2024-05-08T17:05:54.612 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 123 M 2024-05-08T17:05:54.612 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:05:54.917 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 14 kB/s | 1.7 kB 00:00 2024-05-08T17:05:55.059 INFO:teuthology.orchestra.run.smithi183.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.4 MB/s | 484 kB 00:00 2024-05-08T17:05:55.125 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:01 ago on Wed 08 May 2024 05:05:54 PM UTC. 2024-05-08T17:05:55.510 INFO:teuthology.orchestra.run.smithi183.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 20 MB/s | 15 MB 00:00 2024-05-08T17:05:56.651 INFO:teuthology.orchestra.run.smithi183.stdout:(3/4): kernel-modules-core-5.14.0-444.el9.x86_6 18 MB/s | 28 MB 00:01 2024-05-08T17:05:57.016 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T17:05:57.016 INFO:teuthology.orchestra.run.smithi154.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T17:05:57.112 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:05:57.122 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout:Install 4 Packages 2024-05-08T17:05:57.123 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:05:57.124 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 78 M 2024-05-08T17:05:57.124 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 123 M 2024-05-08T17:05:57.124 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:05:57.177 INFO:teuthology.orchestra.run.smithi183.stdout:(4/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 14 MB/s | 34 MB 00:02 2024-05-08T17:05:57.177 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:05:57.177 INFO:teuthology.orchestra.run.smithi183.stdout:Total 30 MB/s | 78 MB 00:02 2024-05-08T17:05:57.569 INFO:teuthology.orchestra.run.smithi154.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 1.6 MB/s | 484 kB 00:00 2024-05-08T17:05:57.856 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:05:58.186 INFO:teuthology.orchestra.run.smithi154.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 17 MB/s | 15 MB 00:00 2024-05-08T17:05:58.199 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:05:58.199 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:05:58.811 INFO:teuthology.orchestra.run.smithi154.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 22 MB/s | 34 MB 00:01 2024-05-08T17:05:59.078 INFO:teuthology.orchestra.run.smithi154.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 19 MB/s | 28 MB 00:01 2024-05-08T17:05:59.078 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:05:59.078 INFO:teuthology.orchestra.run.smithi154.stdout:Total 40 MB/s | 78 MB 00:01 2024-05-08T17:05:59.127 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:05:59.128 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:05:59.769 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:06:00.123 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:06:00.123 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:06:01.051 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:06:01.051 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:06:01.535 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:06:02.161 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:06:02.285 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:03.471 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:06:04.119 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:06:04.204 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:04.953 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:05.086 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:06.906 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:07.043 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:09.827 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:09.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:11.800 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:14.690 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:16.592 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:44.082 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:44.120 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:06:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:45.239 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:45.239 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:06:45.240 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:06:45.603 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:06:45.684 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.684 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:06:45.684 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T17:06:45.685 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T17:06:45.685 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T17:06:45.685 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2024-05-08T17:06:45.685 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi183.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T17:06:45.685 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:06:45.735 INFO:teuthology.orchestra.run.smithi183.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:45.735 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:06:45.736 DEBUG:teuthology.orchestra.run.smithi183:> sudo rpm -qi grub2-tools 2024-05-08T17:06:45.773 INFO:teuthology.orchestra.run.smithi183.stdout:Name : grub2-tools 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Epoch : 1 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Version : 2.06 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Release : 68.el9 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Architecture: x86_64 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Group : Unspecified 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Size : 8264254 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:License : GPLv3+ 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Packager : builder@centos.org 2024-05-08T17:06:45.774 INFO:teuthology.orchestra.run.smithi183.stdout:Vendor : CentOS 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:Summary : Support tools for GRUB. 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:Description : 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:hardware devices. 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:06:45.775 INFO:teuthology.orchestra.run.smithi183.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T17:06:45.776 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T17:06:45.776 INFO:teuthology.task.kernel:Updating grub on smithi183 to boot 5.14.0-444.el9.x86_64 2024-05-08T17:06:45.776 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T17:06:46.036 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:46.075 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:46.349 INFO:teuthology.orchestra.run.smithi183.stderr:Generating grub configuration file ... 2024-05-08T17:06:46.438 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:46.439 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T17:06:46.439 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T17:06:46.439 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:06:47.301 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:06:47.643 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:06:47.714 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.714 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:06:47.715 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T17:06:47.715 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T17:06:47.715 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T17:06:47.715 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2024-05-08T17:06:47.715 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi154.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T17:06:47.716 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T17:06:47.767 INFO:teuthology.orchestra.run.smithi154.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T17:06:47.767 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T17:06:47.767 DEBUG:teuthology.orchestra.run.smithi154:> sudo rpm -qi grub2-tools 2024-05-08T17:06:47.773 INFO:teuthology.orchestra.run.smithi183.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Name : grub2-tools 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Epoch : 1 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Version : 2.06 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Release : 68.el9 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Architecture: x86_64 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Group : Unspecified 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Size : 8264254 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:License : GPLv3+ 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T17:06:47.798 INFO:teuthology.orchestra.run.smithi154.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Packager : builder@centos.org 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Vendor : CentOS 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Summary : Support tools for GRUB. 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:Description : 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:hardware devices. 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:06:47.799 INFO:teuthology.orchestra.run.smithi154.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T17:06:47.800 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T17:06:47.800 INFO:teuthology.task.kernel:Updating grub on smithi154 to boot 5.14.0-444.el9.x86_64 2024-05-08T17:06:47.800 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T17:06:47.807 INFO:teuthology.orchestra.run.smithi183.stderr:done 2024-05-08T17:06:47.812 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-05-08T17:06:47.828 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.Sn7beKSaTK 2024-05-08T17:06:47.828 DEBUG:teuthology.orchestra.run.smithi183:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.Sn7beKSaTK 2024-05-08T17:06:47.907 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.Sn7beKSaTK 2024-05-08T17:06:48.048 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.Sn7beKSaTK is 6KB 2024-05-08T17:06:48.096 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.Sn7beKSaTK 2024-05-08T17:06:48.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T17:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T17:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T17:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T17:06:48.181 INFO:teuthology.orchestra.run.smithi183.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T17:06:48.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T17:06:48.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo shutdown -r now 2024-05-08T17:06:48.426 INFO:teuthology.orchestra.run.smithi154.stderr:Generating grub configuration file ... 2024-05-08T17:06:49.778 INFO:teuthology.orchestra.run.smithi154.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T17:06:49.810 INFO:teuthology.orchestra.run.smithi154.stderr:done 2024-05-08T17:06:49.812 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-05-08T17:06:49.827 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.XPFFFM1ICD 2024-05-08T17:06:49.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XPFFFM1ICD 2024-05-08T17:06:49.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.XPFFFM1ICD 2024-05-08T17:06:50.035 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.XPFFFM1ICD is 6KB 2024-05-08T17:06:50.084 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.XPFFFM1ICD 2024-05-08T17:06:50.098 DEBUG:teuthology.orchestra.run.smithi154:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T17:06:50.167 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T17:06:50.167 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T17:06:50.168 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T17:06:50.168 INFO:teuthology.orchestra.run.smithi154.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T17:06:50.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T17:06:50.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo shutdown -r now 2024-05-08T17:07:18.371 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2024-05-08T17:07:18.371 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:07:18.373 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:07:20.348 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2024-05-08T17:07:20.349 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:07:20.349 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:07:36.863 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.183 2024-05-08T17:07:45.870 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:07:45.871 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:07:55.038 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-05-08T17:08:04.043 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:08:04.044 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:08:22.435 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.154 2024-05-08T17:08:34.436 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:08:34.437 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:08:37.798 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-05-08T17:08:38.288 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:08:38.288 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-444.el9.x86_64"... 2024-05-08T17:08:38.288 DEBUG:teuthology.orchestra.run.smithi154:> uname -r 2024-05-08T17:08:38.303 INFO:teuthology.orchestra.run.smithi154.stdout:5.14.0-444.el9.x86_64 2024-05-08T17:08:38.303 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T17:08:38.303 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T17:08:38.303 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T17:08:39.304 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-05-08T17:08:39.304 DEBUG:teuthology.orchestra.run.smithi154:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T17:08:39.407 INFO:teuthology.orchestra.run.smithi154.stdout:ttyS1 2024-05-08T17:08:39.444 DEBUG:teuthology.parallel:result is None 2024-05-08T17:08:45.874 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T17:08:57.875 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:08:57.876 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:08:58.290 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-05-08T17:08:58.667 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:08:58.667 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "5.14.0-444.el9.x86_64"... 2024-05-08T17:08:58.667 DEBUG:teuthology.orchestra.run.smithi183:> uname -r 2024-05-08T17:08:58.684 INFO:teuthology.orchestra.run.smithi183.stdout:5.14.0-444.el9.x86_64 2024-05-08T17:08:58.685 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T17:08:58.685 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T17:08:58.685 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T17:08:59.685 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2024-05-08T17:08:59.686 DEBUG:teuthology.orchestra.run.smithi183:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-05-08T17:08:59.762 INFO:teuthology.orchestra.run.smithi183.stdout:ttyS1 2024-05-08T17:08:59.798 DEBUG:teuthology.parallel:result is None 2024-05-08T17:08:59.799 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T17:08:59.808 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T17:08:59.808 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T17:08:59.811 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T17:08:59.830 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T17:08:59.838 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T17:08:59.854 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T17:08:59.855 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T17:08:59.872 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T17:08:59.980 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T17:08:59.988 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T17:08:59.988 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T17:08:59.991 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T17:09:00.020 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:09:00.025 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:09:00.032 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:09:00.036 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T17:09:00.038 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T17:09:00.046 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T17:09:00.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T17:09:00.076 DEBUG:teuthology.orchestra.run.smithi183:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T17:09:00.107 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T17:09:00.117 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T17:09:00.118 DEBUG:teuthology.orchestra.run.smithi154:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T17:09:00.144 DEBUG:teuthology.orchestra.run.smithi183:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T17:09:00.164 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:09:00.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:09:00.288 DEBUG:teuthology.orchestra.run.smithi154:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:09:00.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:09:00.414 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:09:00.414 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T17:09:00.478 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:09:00.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T17:09:00.580 DEBUG:teuthology.orchestra.run.smithi183:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:09:00.665 DEBUG:teuthology.orchestra.run.smithi183:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T17:09:00.708 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:09:00.708 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T17:09:00.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo service rsyslog restart 2024-05-08T17:09:00.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo service rsyslog restart 2024-05-08T17:09:00.816 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:09:00.858 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:09:01.273 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T17:09:01.284 INFO:teuthology.task.internal:Starting timer... 2024-05-08T17:09:01.284 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T17:09:01.297 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T17:09:01.307 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-05-08T17:09:01.307 DEBUG:teuthology.orchestra.run.smithi154:> sudo service auditd rotate 2024-05-08T17:09:01.375 INFO:teuthology.orchestra.run.smithi154.stdout:Rotating logs: 2024-05-08T17:09:01.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo service auditd rotate 2024-05-08T17:09:01.446 INFO:teuthology.orchestra.run.smithi183.stdout:Rotating logs: 2024-05-08T17:09:01.448 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T17:09:01.448 DEBUG:teuthology.orchestra.run.smithi154:> /usr/sbin/getenforce 2024-05-08T17:09:01.493 INFO:teuthology.orchestra.run.smithi154.stdout:Permissive 2024-05-08T17:09:01.493 DEBUG:teuthology.orchestra.run.smithi183:> /usr/sbin/getenforce 2024-05-08T17:09:01.527 INFO:teuthology.orchestra.run.smithi183.stdout:Permissive 2024-05-08T17:09:01.528 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi154.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi183.front.sepia.ceph.com': 'permissive'} 2024-05-08T17:09:01.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:09:01.554 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:09:01.554 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:09:01.596 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:09:01.596 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T17:09:01.596 DEBUG:teuthology.orchestra.run.smithi154:> sudo /usr/sbin/setenforce permissive 2024-05-08T17:09:01.622 DEBUG:teuthology.orchestra.run.smithi183:> sudo /usr/sbin/setenforce permissive 2024-05-08T17:09:01.661 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T17:09:01.671 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T17:09:01.677 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T17:09:01.801 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T17:09:01.823 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T17:09:01.824 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi154.front.sepia.ceph.com,smithi183.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T17:13:02.280 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi154.front.sepia.ceph.com'), Remote(name='ubuntu@smithi183.front.sepia.ceph.com')] 2024-05-08T17:13:02.281 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:13:02.283 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi154.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:13:02.358 DEBUG:teuthology.orchestra.run.smithi154:> true 2024-05-08T17:13:02.438 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi154.front.sepia.ceph.com' 2024-05-08T17:13:02.439 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:13:02.439 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi183.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T17:13:02.517 DEBUG:teuthology.orchestra.run.smithi183:> true 2024-05-08T17:13:02.606 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi183.front.sepia.ceph.com' 2024-05-08T17:13:02.606 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T17:13:02.618 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T17:13:02.618 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T17:13:02.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T17:13:02.621 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T17:13:02.621 DEBUG:teuthology.orchestra.run.smithi183:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T17:13:02.650 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T17:13:02.664 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T17:13:02.675 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T17:13:02.689 INFO:teuthology.orchestra.run.smithi154.stderr:sudo: ntpd: command not found 2024-05-08T17:13:02.689 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T17:13:02.702 INFO:teuthology.orchestra.run.smithi154.stdout:506 Cannot talk to daemon 2024-05-08T17:13:02.713 INFO:teuthology.orchestra.run.smithi183.stderr:sudo: ntpd: command not found 2024-05-08T17:13:02.714 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T17:13:02.724 INFO:teuthology.orchestra.run.smithi183.stdout:506 Cannot talk to daemon 2024-05-08T17:13:02.727 INFO:teuthology.orchestra.run.smithi154.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T17:13:02.738 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T17:13:02.752 INFO:teuthology.orchestra.run.smithi183.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T17:13:02.785 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.789 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.804 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.808 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:13:02.809 INFO:teuthology.run_tasks:Running task install... 2024-05-08T17:13:02.820 DEBUG:teuthology.task.install:project ceph 2024-05-08T17:13:02.820 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}} 2024-05-08T17:13:02.820 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'} 2024-05-08T17:13:02.820 INFO:teuthology.task.install:Using flavor: default 2024-05-08T17:13:02.826 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T17:13:02.826 INFO:teuthology.task.install:extra packages: [] 2024-05-08T17:13:02.827 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'tag': None, 'wait_for_package': False} 2024-05-08T17:13:02.828 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T17:13:02.830 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'tag': None, 'wait_for_package': False} 2024-05-08T17:13:02.830 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T17:13:02.976 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/ 2024-05-08T17:13:02.976 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2517.g95a4ad3c 2024-05-08T17:13:03.063 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/ 2024-05-08T17:13:03.063 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2517.g95a4ad3c 2024-05-08T17:13:03.111 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T17:13:03.112 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:13:03.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T17:13:03.138 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T17:13:03.139 DEBUG:teuthology.orchestra.run.smithi154:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95a4ad3c90f2f12f920e87725ad8d9002f948722/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T17:13:03.195 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-lflores-testing-2-2024-05-07-1606-squid/95a4ad3c90f2f12f920e87725ad8d9002f948722/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T17:13:03.195 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:13:03.195 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T17:13:03.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T17:13:03.222 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-05-08T17:13:03.223 DEBUG:teuthology.orchestra.run.smithi183:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/95a4ad3c90f2f12f920e87725ad8d9002f948722/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T17:13:03.280 DEBUG:teuthology.orchestra.run.smithi154:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T17:13:03.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T17:13:03.345 INFO:teuthology.orchestra.run.smithi154.stdout:check_obsoletes = 1 2024-05-08T17:13:03.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum clean all 2024-05-08T17:13:03.363 DEBUG:teuthology.orchestra.run.smithi183:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T17:13:03.428 INFO:teuthology.orchestra.run.smithi183.stdout:check_obsoletes = 1 2024-05-08T17:13:03.429 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum clean all 2024-05-08T17:13:03.586 INFO:teuthology.orchestra.run.smithi154.stdout:54 files removed 2024-05-08T17:13:03.614 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-radosgw 2024-05-08T17:13:03.667 INFO:teuthology.orchestra.run.smithi183.stdout:54 files removed 2024-05-08T17:13:03.695 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-radosgw 2024-05-08T17:13:04.344 INFO:teuthology.orchestra.run.smithi154.stdout:ceph packages for x86_64 183 kB/s | 83 kB 00:00 2024-05-08T17:13:04.436 INFO:teuthology.orchestra.run.smithi183.stdout:ceph packages for x86_64 179 kB/s | 83 kB 00:00 2024-05-08T17:13:04.820 INFO:teuthology.orchestra.run.smithi183.stdout:ceph noarch packages 35 kB/s | 11 kB 00:00 2024-05-08T17:13:04.895 INFO:teuthology.orchestra.run.smithi154.stdout:ceph noarch packages 23 kB/s | 11 kB 00:00 2024-05-08T17:13:05.262 INFO:teuthology.orchestra.run.smithi183.stdout:ceph source packages 4.8 kB/s | 1.9 kB 00:00 2024-05-08T17:13:05.320 INFO:teuthology.orchestra.run.smithi154.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-05-08T17:13:06.537 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - BaseOS 6.9 MB/s | 8.1 MB 00:01 2024-05-08T17:13:08.431 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - AppStream 23 MB/s | 19 MB 00:00 2024-05-08T17:13:10.570 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - BaseOS 1.5 MB/s | 8.1 MB 00:05 2024-05-08T17:13:13.966 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - CRB 3.1 MB/s | 6.2 MB 00:01 2024-05-08T17:13:15.137 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - AppStream 5.5 MB/s | 19 MB 00:03 2024-05-08T17:13:15.516 INFO:teuthology.orchestra.run.smithi154.stdout:CentOS Stream 9 - Extras packages 76 kB/s | 16 kB 00:00 2024-05-08T17:13:18.634 INFO:teuthology.orchestra.run.smithi154.stdout:Extra Packages for Enterprise Linux 7.1 MB/s | 21 MB 00:02 2024-05-08T17:13:20.929 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - CRB 2.8 MB/s | 6.2 MB 00:02 2024-05-08T17:13:22.538 INFO:teuthology.orchestra.run.smithi183.stdout:CentOS Stream 9 - Extras packages 60 kB/s | 16 kB 00:00 2024-05-08T17:13:23.794 INFO:teuthology.orchestra.run.smithi154.stdout:lab-extras 43 kB/s | 1.7 kB 00:00 2024-05-08T17:13:25.781 INFO:teuthology.orchestra.run.smithi183.stdout:Extra Packages for Enterprise Linux 6.8 MB/s | 21 MB 00:03 2024-05-08T17:13:26.159 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:13:26.170 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:26.170 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:13:26.170 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:26.170 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 10 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrading: 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: librados2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.4 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.4 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 21 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 25 k 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T17:13:26.171 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 745 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 501 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.3 M 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 45 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 135 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 162 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T17:13:26.172 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 320 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 299 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 99 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout:Installing weak dependencies: 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout:Install 30 Packages 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout:Upgrade 2 Packages 2024-05-08T17:13:26.173 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:26.174 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 60 M 2024-05-08T17:13:26.174 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:13:28.060 INFO:teuthology.orchestra.run.smithi154.stdout:(1/32): ceph-base-19.0.0-2517.g95a4ad3c.el9.x86 3.7 MB/s | 5.4 MB 00:01 2024-05-08T17:13:28.152 INFO:teuthology.orchestra.run.smithi154.stdout:(2/32): ceph-selinux-19.0.0-2517.g95a4ad3c.el9. 274 kB/s | 25 kB 00:00 2024-05-08T17:13:28.278 INFO:teuthology.orchestra.run.smithi154.stdout:(3/32): libcephfs2-19.0.0-2517.g95a4ad3c.el9.x8 5.8 MB/s | 745 kB 00:00 2024-05-08T17:13:28.419 INFO:teuthology.orchestra.run.smithi154.stdout:(4/32): libradosstriper1-19.0.0-2517.g95a4ad3c. 3.5 MB/s | 501 kB 00:00 2024-05-08T17:13:28.611 INFO:teuthology.orchestra.run.smithi154.stdout:(5/32): ceph-radosgw-19.0.0-2517.g95a4ad3c.el9. 5.1 MB/s | 10 MB 00:01 2024-05-08T17:13:28.695 INFO:teuthology.orchestra.run.smithi154.stdout:(6/32): python3-ceph-argparse-19.0.0-2517.g95a4 543 kB/s | 45 kB 00:00 2024-05-08T17:13:28.770 INFO:teuthology.orchestra.run.smithi154.stdout:(7/32): python3-ceph-common-19.0.0-2517.g95a4ad 1.8 MB/s | 135 kB 00:00 2024-05-08T17:13:28.854 INFO:teuthology.orchestra.run.smithi154.stdout:(8/32): python3-cephfs-19.0.0-2517.g95a4ad3c.el 1.9 MB/s | 162 kB 00:00 2024-05-08T17:13:28.955 INFO:teuthology.orchestra.run.smithi154.stdout:(9/32): python3-rados-19.0.0-2517.g95a4ad3c.el9 3.1 MB/s | 320 kB 00:00 2024-05-08T17:13:29.055 INFO:teuthology.orchestra.run.smithi154.stdout:(10/32): python3-rbd-19.0.0-2517.g95a4ad3c.el9. 2.9 MB/s | 299 kB 00:00 2024-05-08T17:13:29.254 INFO:teuthology.orchestra.run.smithi154.stdout:(11/32): librgw2-19.0.0-2517.g95a4ad3c.el9.x86_ 6.4 MB/s | 5.3 MB 00:00 2024-05-08T17:13:29.281 INFO:teuthology.orchestra.run.smithi154.stdout:(12/32): python3-rgw-19.0.0-2517.g95a4ad3c.el9. 441 kB/s | 99 kB 00:00 2024-05-08T17:13:29.523 INFO:teuthology.orchestra.run.smithi154.stdout:(13/32): ceph-common-19.0.0-2517.g95a4ad3c.el9. 7.2 MB/s | 21 MB 00:02 2024-05-08T17:13:29.565 INFO:teuthology.orchestra.run.smithi154.stdout:(14/32): boost-program-options-1.75.0-8.el9.x86 344 kB/s | 107 kB 00:00 2024-05-08T17:13:29.640 INFO:teuthology.orchestra.run.smithi154.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 388 kB/s | 45 kB 00:00 2024-05-08T17:13:29.673 INFO:teuthology.orchestra.run.smithi154.stdout:(16/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 408 kB/s | 160 kB 00:00 2024-05-08T17:13:29.707 INFO:teuthology.orchestra.run.smithi154.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-05-08T17:13:29.724 INFO:teuthology.orchestra.run.smithi154.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 842 kB/s | 42 kB 00:00 2024-05-08T17:13:29.758 INFO:teuthology.orchestra.run.smithi154.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-05-08T17:13:29.783 INFO:teuthology.orchestra.run.smithi154.stdout:(20/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.0 MB/s | 292 kB 00:00 2024-05-08T17:13:29.800 INFO:teuthology.orchestra.run.smithi154.stdout:(21/32): qatlib-service-24.02.0-1.el9.x86_64.rp 475 kB/s | 36 kB 00:00 2024-05-08T17:13:29.817 INFO:teuthology.orchestra.run.smithi154.stdout:(22/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 805 kB/s | 47 kB 00:00 2024-05-08T17:13:29.934 INFO:teuthology.orchestra.run.smithi154.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 223 kB/s | 26 kB 00:00 2024-05-08T17:13:30.026 INFO:teuthology.orchestra.run.smithi154.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 531 kB/s | 49 kB 00:00 2024-05-08T17:13:30.051 INFO:teuthology.orchestra.run.smithi154.stdout:(25/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.1 MB/s | 309 kB 00:00 2024-05-08T17:13:30.202 INFO:teuthology.orchestra.run.smithi154.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 384 kB/s | 67 kB 00:00 2024-05-08T17:13:30.243 INFO:teuthology.orchestra.run.smithi154.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 839 kB 00:00 2024-05-08T17:13:30.310 INFO:teuthology.orchestra.run.smithi154.stdout:(28/32): libarrow-9.0.0-11.el9.x86_64.rpm 8.7 MB/s | 4.4 MB 00:00 2024-05-08T17:13:30.336 INFO:teuthology.orchestra.run.smithi154.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 1.4 MB/s | 192 kB 00:00 2024-05-08T17:13:30.377 INFO:teuthology.orchestra.run.smithi154.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-08T17:13:31.015 INFO:teuthology.orchestra.run.smithi183.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-05-08T17:13:31.094 INFO:teuthology.orchestra.run.smithi154.stdout:(31/32): librados2-19.0.0-2517.g95a4ad3c.el9.x8 4.3 MB/s | 3.4 MB 00:00 2024-05-08T17:13:31.245 INFO:teuthology.orchestra.run.smithi154.stdout:(32/32): librbd1-19.0.0-2517.g95a4ad3c.el9.x86_ 3.5 MB/s | 3.1 MB 00:00 2024-05-08T17:13:31.247 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:13:31.247 INFO:teuthology.orchestra.run.smithi154.stdout:Total 12 MB/s | 60 MB 00:05 2024-05-08T17:13:31.355 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:13:31.392 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:13:31.392 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:13:32.016 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:13:32.016 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:13:33.348 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 10 M 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrading: 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout: librados2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.4 M 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T17:13:33.402 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.4 M 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 21 M 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 25 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 745 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 501 k 2024-05-08T17:13:33.403 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2 x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 5.3 M 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 45 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 135 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 162 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 320 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 299 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 99 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T17:13:33.404 INFO:teuthology.orchestra.run.smithi183.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:Installing weak dependencies: 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:Install 30 Packages 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:Upgrade 2 Packages 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:33.405 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 60 M 2024-05-08T17:13:33.406 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:13:33.410 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:13:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T17:13:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T17:13:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T17:13:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T17:13:33.583 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T17:13:33.775 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T17:13:33.839 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T17:13:34.165 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T17:13:34.268 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:34.352 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:34.439 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 8/34 2024-05-08T17:13:34.470 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T17:13:34.517 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T17:13:34.553 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T17:13:34.597 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 11/34 2024-05-08T17:13:34.665 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T17:13:34.689 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T17:13:34.753 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T17:13:34.788 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T17:13:34.900 INFO:teuthology.orchestra.run.smithi183.stdout:(1/32): ceph-radosgw-19.0.0-2517.g95a4ad3c.el9. 8.3 MB/s | 10 MB 00:01 2024-05-08T17:13:34.904 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T17:13:34.938 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T17:13:34.950 INFO:teuthology.orchestra.run.smithi183.stdout:(2/32): ceph-selinux-19.0.0-2517.g95a4ad3c.el9. 506 kB/s | 25 kB 00:00 2024-05-08T17:13:34.981 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T17:13:35.012 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T17:13:35.041 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T17:13:35.074 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T17:13:35.225 INFO:teuthology.orchestra.run.smithi183.stdout:(3/32): ceph-common-19.0.0-2517.g95a4ad3c.el9.x 14 MB/s | 21 MB 00:01 2024-05-08T17:13:35.250 INFO:teuthology.orchestra.run.smithi183.stdout:(4/32): libcephfs2-19.0.0-2517.g95a4ad3c.el9.x8 2.4 MB/s | 745 kB 00:00 2024-05-08T17:13:35.284 INFO:teuthology.orchestra.run.smithi183.stdout:(5/32): libradosstriper1-19.0.0-2517.g95a4ad3c. 8.4 MB/s | 501 kB 00:00 2024-05-08T17:13:35.334 INFO:teuthology.orchestra.run.smithi183.stdout:(6/32): python3-ceph-argparse-19.0.0-2517.g95a4 902 kB/s | 45 kB 00:00 2024-05-08T17:13:35.385 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T17:13:35.393 INFO:teuthology.orchestra.run.smithi183.stdout:(7/32): python3-ceph-common-19.0.0-2517.g95a4ad 2.3 MB/s | 135 kB 00:00 2024-05-08T17:13:35.428 INFO:teuthology.orchestra.run.smithi154.stdout: Upgrading : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T17:13:35.461 INFO:teuthology.orchestra.run.smithi183.stdout:(8/32): python3-cephfs-19.0.0-2517.g95a4ad3c.el 2.4 MB/s | 162 kB 00:00 2024-05-08T17:13:35.502 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T17:13:35.534 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 23/34 2024-05-08T17:13:35.535 INFO:teuthology.orchestra.run.smithi183.stdout:(9/32): python3-rados-19.0.0-2517.g95a4ad3c.el9 4.2 MB/s | 320 kB 00:00 2024-05-08T17:13:35.570 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T17:13:35.602 INFO:teuthology.orchestra.run.smithi183.stdout:(10/32): librgw2-19.0.0-2517.g95a4ad3c.el9.x86_ 15 MB/s | 5.3 MB 00:00 2024-05-08T17:13:35.627 INFO:teuthology.orchestra.run.smithi183.stdout:(11/32): python3-rbd-19.0.0-2517.g95a4ad3c.el9. 3.2 MB/s | 299 kB 00:00 2024-05-08T17:13:35.653 INFO:teuthology.orchestra.run.smithi183.stdout:(12/32): python3-rgw-19.0.0-2517.g95a4ad3c.el9. 1.9 MB/s | 99 kB 00:00 2024-05-08T17:13:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T17:13:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T17:13:36.078 INFO:teuthology.orchestra.run.smithi183.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 237 kB/s | 107 kB 00:00 2024-05-08T17:13:36.136 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T17:13:36.195 INFO:teuthology.orchestra.run.smithi183.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 296 kB/s | 160 kB 00:00 2024-05-08T17:13:36.213 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 27/34 2024-05-08T17:13:36.220 INFO:teuthology.orchestra.run.smithi183.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 319 kB/s | 45 kB 00:00 2024-05-08T17:13:36.250 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 28/34 2024-05-08T17:13:36.537 INFO:teuthology.orchestra.run.smithi183.stdout:(16/32): ceph-base-19.0.0-2517.g95a4ad3c.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-05-08T17:13:36.562 INFO:teuthology.orchestra.run.smithi183.stdout:(17/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 856 kB/s | 292 kB 00:00 2024-05-08T17:13:36.596 INFO:teuthology.orchestra.run.smithi183.stdout:(18/32): librdkafka-1.6.1-102.el9.x86_64.rpm 1.6 MB/s | 662 kB 00:00 2024-05-08T17:13:36.671 INFO:teuthology.orchestra.run.smithi183.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.0 MB/s | 221 kB 00:00 2024-05-08T17:13:36.696 INFO:teuthology.orchestra.run.smithi183.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 358 kB/s | 36 kB 00:00 2024-05-08T17:13:36.780 INFO:teuthology.orchestra.run.smithi183.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 437 kB/s | 47 kB 00:00 2024-05-08T17:13:36.880 INFO:teuthology.orchestra.run.smithi183.stdout:(22/32): python3-prettytable-0.7.2-27.el9.noarc 123 kB/s | 42 kB 00:00 2024-05-08T17:13:36.939 INFO:teuthology.orchestra.run.smithi183.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 1.2 MB/s | 309 kB 00:00 2024-05-08T17:13:36.997 INFO:teuthology.orchestra.run.smithi183.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 832 kB/s | 49 kB 00:00 2024-05-08T17:13:37.023 INFO:teuthology.orchestra.run.smithi183.stdout:(25/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 183 kB/s | 26 kB 00:00 2024-05-08T17:13:37.057 INFO:teuthology.orchestra.run.smithi183.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-08T17:13:37.123 INFO:teuthology.orchestra.run.smithi183.stdout:(27/32): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-08T17:13:37.207 INFO:teuthology.orchestra.run.smithi183.stdout:(28/32): libarrow-9.0.0-11.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-08T17:13:37.307 INFO:teuthology.orchestra.run.smithi183.stdout:(29/32): thrift-0.15.0-2.el9.x86_64.rpm 8.6 MB/s | 1.6 MB 00:00 2024-05-08T17:13:37.341 INFO:teuthology.orchestra.run.smithi183.stdout:(30/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.6 MB/s | 839 kB 00:00 2024-05-08T17:13:37.583 INFO:teuthology.orchestra.run.smithi183.stdout:(31/32): librbd1-19.0.0-2517.g95a4ad3c.el9.x86_ 11 MB/s | 3.1 MB 00:00 2024-05-08T17:13:38.041 INFO:teuthology.orchestra.run.smithi183.stdout:(32/32): librados2-19.0.0-2517.g95a4ad3c.el9.x8 4.1 MB/s | 3.4 MB 00:00 2024-05-08T17:13:38.044 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:13:38.044 INFO:teuthology.orchestra.run.smithi183.stdout:Total 13 MB/s | 60 MB 00:04 2024-05-08T17:13:38.152 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:13:38.187 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:13:38.187 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:13:38.530 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:38.564 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:38.799 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:13:38.799 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:13:39.173 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:39.218 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T17:13:39.310 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T17:13:39.310 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T17:13:39.311 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T17:13:39.311 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:39.358 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:40.230 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:13:40.315 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-05-08T17:13:40.346 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-05-08T17:13:40.372 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-05-08T17:13:40.394 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T17:13:40.428 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-05-08T17:13:40.613 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T17:13:40.671 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-05-08T17:13:41.047 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-05-08T17:13:41.106 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:41.182 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:41.272 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 8/34 2024-05-08T17:13:41.300 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T17:13:41.363 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 9/34 2024-05-08T17:13:41.400 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-05-08T17:13:41.452 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 11/34 2024-05-08T17:13:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T17:13:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T17:13:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 13/34 2024-05-08T17:13:41.630 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-05-08T17:13:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-05-08T17:13:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-05-08T17:13:41.836 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-05-08T17:13:41.866 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-05-08T17:13:41.895 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-05-08T17:13:41.937 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-05-08T17:13:42.256 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-05-08T17:13:42.291 INFO:teuthology.orchestra.run.smithi183.stdout: Upgrading : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T17:13:42.390 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 22/34 2024-05-08T17:13:42.421 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 23/34 2024-05-08T17:13:42.457 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-05-08T17:13:42.939 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-05-08T17:13:42.969 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T17:13:43.024 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 26/34 2024-05-08T17:13:43.100 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 27/34 2024-05-08T17:13:43.129 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 28/34 2024-05-08T17:13:45.454 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:45.492 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:46.138 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 29/34 2024-05-08T17:13:46.188 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T17:13:46.280 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 30/34 2024-05-08T17:13:46.280 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T17:13:46.280 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T17:13:46.280 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:46.314 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /sys 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /proc 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /mnt 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /var/tmp 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /home 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /root 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout:skipping the directory /tmp 2024-05-08T17:13:49.030 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:49.938 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T17:13:49.963 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T17:13:49.963 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:13:49.963 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T17:13:49.964 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T17:13:49.964 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T17:13:49.964 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:49.965 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T17:13:50.031 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T17:13:50.032 INFO:teuthology.orchestra.run.smithi154.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 6/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 8/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 9/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 10/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 11/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 13/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T17:13:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T17:13:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T17:13:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/34 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout:Upgraded: 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.208 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.209 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:52.210 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:13:52.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-test 2024-05-08T17:13:53.102 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:30 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:13:53.223 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 48 M 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:13:53.224 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 163 k 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout:Install 4 Packages 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:13:53.225 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 49 M 2024-05-08T17:13:53.226 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 207 M 2024-05-08T17:13:53.226 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:13:53.604 INFO:teuthology.orchestra.run.smithi154.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 983 kB/s | 305 kB 00:00 2024-05-08T17:13:53.663 INFO:teuthology.orchestra.run.smithi154.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T17:13:53.703 INFO:teuthology.orchestra.run.smithi154.stdout:(3/4): libcephsqlite-19.0.0-2517.g95a4ad3c.el9. 398 kB/s | 163 kB 00:00 2024-05-08T17:13:55.755 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:55.755 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /sys 2024-05-08T17:13:55.755 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /proc 2024-05-08T17:13:55.755 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /mnt 2024-05-08T17:13:55.756 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /var/tmp 2024-05-08T17:13:55.756 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /home 2024-05-08T17:13:55.756 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /root 2024-05-08T17:13:55.756 INFO:teuthology.orchestra.run.smithi183.stdout:skipping the directory /tmp 2024-05-08T17:13:55.756 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:55.804 INFO:teuthology.orchestra.run.smithi154.stdout:(4/4): ceph-test-19.0.0-2517.g95a4ad3c.el9.x86_ 19 MB/s | 48 MB 00:02 2024-05-08T17:13:55.806 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:13:55.807 INFO:teuthology.orchestra.run.smithi154.stdout:Total 19 MB/s | 49 MB 00:02 2024-05-08T17:13:55.835 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:13:55.856 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:13:55.856 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:13:56.221 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:13:56.221 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:13:56.732 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T17:13:56.757 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/34 2024-05-08T17:13:56.757 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:13:56.757 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T17:13:56.758 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T17:13:56.758 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T17:13:56.758 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:56.758 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:13:56.759 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T17:13:56.839 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T17:13:56.843 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-05-08T17:13:56.843 INFO:teuthology.orchestra.run.smithi183.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:56.878 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T17:13:56.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T17:13:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/34 2024-05-08T17:13:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/34 2024-05-08T17:13:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/34 2024-05-08T17:13:58.005 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_6 6/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 7/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9. 8/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x8 9/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 10/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 11/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 12/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 13/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-05-08T17:13:58.006 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-05-08T17:13:58.007 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 31/34 2024-05-08T17:13:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-05-08T17:13:58.008 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/34 2024-05-08T17:13:58.829 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-05-08T17:13:58.829 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout:Upgraded: 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-radosgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-selinux-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T17:13:58.830 INFO:teuthology.orchestra.run.smithi183.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: libradosstriper1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: librgw2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-argparse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ceph-common-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T17:13:58.831 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:58.832 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:13:59.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-test 2024-05-08T17:13:59.772 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:29 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:13:59.893 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 48 M 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 163 k 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T17:13:59.894 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:Install 4 Packages 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 49 M 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 207 M 2024-05-08T17:13:59.895 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:00.172 INFO:teuthology.orchestra.run.smithi183.stdout:(1/4): libcephsqlite-19.0.0-2517.g95a4ad3c.el9. 687 kB/s | 163 kB 00:00 2024-05-08T17:14:00.523 INFO:teuthology.orchestra.run.smithi183.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 182 kB/s | 64 kB 00:00 2024-05-08T17:14:00.548 INFO:teuthology.orchestra.run.smithi183.stdout:(3/4): socat-1.7.4.1-5.el9.x86_64.rpm 497 kB/s | 305 kB 00:00 2024-05-08T17:14:01.482 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T17:14:01.573 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T17:14:02.415 INFO:teuthology.orchestra.run.smithi183.stdout:(4/4): ceph-test-19.0.0-2517.g95a4ad3c.el9.x86_ 19 MB/s | 48 MB 00:02 2024-05-08T17:14:02.417 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:02.417 INFO:teuthology.orchestra.run.smithi183.stdout:Total 19 MB/s | 49 MB 00:02 2024-05-08T17:14:02.446 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:02.466 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:02.467 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:02.833 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:02.833 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:03.365 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:03.441 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-05-08T17:14:03.481 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-05-08T17:14:03.508 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T17:14:04.058 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T17:14:04.058 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/4 2024-05-08T17:14:04.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/4 2024-05-08T17:14:04.059 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:04.325 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:04.589 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph 2024-05-08T17:14:05.161 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:42 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:05.284 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout: ceph x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 6.6 k 2024-05-08T17:14:05.286 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 2.4 M 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 1.5 M 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 249 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 4.7 M 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 17 M 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T17:14:05.287 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T17:14:05.288 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout:Install 36 Packages 2024-05-08T17:14:05.289 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:05.290 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 30 M 2024-05-08T17:14:05.290 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 107 M 2024-05-08T17:14:05.290 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:14:06.484 INFO:teuthology.orchestra.run.smithi154.stdout:(1/36): ceph-19.0.0-2517.g95a4ad3c.el9.x86_64.r 27 kB/s | 6.6 kB 00:00 2024-05-08T17:14:06.650 INFO:teuthology.orchestra.run.smithi154.stdout:(2/36): ceph-mds-19.0.0-2517.g95a4ad3c.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-05-08T17:14:06.691 INFO:teuthology.orchestra.run.smithi154.stdout:(3/36): ceph-mgr-19.0.0-2517.g95a4ad3c.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-05-08T17:14:06.825 INFO:teuthology.orchestra.run.smithi154.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2517.g95a4 1.8 MB/s | 249 kB 00:00 2024-05-08T17:14:07.434 INFO:teuthology.orchestra.run.smithi154.stdout:(5/36): ceph-osd-19.0.0-2517.g95a4ad3c.el9.x86_ 21 MB/s | 17 MB 00:00 2024-05-08T17:14:07.459 INFO:teuthology.orchestra.run.smithi154.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 399 kB/s | 253 kB 00:00 2024-05-08T17:14:07.559 INFO:teuthology.orchestra.run.smithi154.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 1.0 MB/s | 106 kB 00:00 2024-05-08T17:14:07.668 INFO:teuthology.orchestra.run.smithi154.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.2 MB/s | 135 kB 00:00 2024-05-08T17:14:07.777 INFO:teuthology.orchestra.run.smithi154.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.1 MB/s | 125 kB 00:00 2024-05-08T17:14:07.902 INFO:teuthology.orchestra.run.smithi154.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.7 MB/s | 215 kB 00:00 2024-05-08T17:14:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/4 2024-05-08T17:14:08.140 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T17:14:08.144 INFO:teuthology.orchestra.run.smithi154.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 780 kB/s | 188 kB 00:00 2024-05-08T17:14:08.252 INFO:teuthology.orchestra.run.smithi154.stdout:(12/36): ceph-mon-19.0.0-2517.g95a4ad3c.el9.x86 2.7 MB/s | 4.7 MB 00:01 2024-05-08T17:14:08.278 INFO:teuthology.orchestra.run.smithi154.stdout:(13/36): python3-mako-1.1.4-6.el9.noarch.rpm 1.3 MB/s | 172 kB 00:00 2024-05-08T17:14:08.336 INFO:teuthology.orchestra.run.smithi154.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 873 kB/s | 51 kB 00:00 2024-05-08T17:14:08.378 INFO:teuthology.orchestra.run.smithi154.stdout:(15/36): python3-cryptography-36.0.1-4.el9.x86_ 1.3 MB/s | 1.2 MB 00:00 2024-05-08T17:14:08.403 INFO:teuthology.orchestra.run.smithi154.stdout:(16/36): python3-markupsafe-1.1.1-12.el9.x86_64 231 kB/s | 35 kB 00:00 2024-05-08T17:14:08.429 INFO:teuthology.orchestra.run.smithi154.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 452 kB/s | 42 kB 00:00 2024-05-08T17:14:08.487 INFO:teuthology.orchestra.run.smithi154.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 205 kB/s | 22 kB 00:00 2024-05-08T17:14:08.746 INFO:teuthology.orchestra.run.smithi154.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 137 kB/s | 43 kB 00:00 2024-05-08T17:14:08.813 INFO:teuthology.orchestra.run.smithi154.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 369 kB/s | 151 kB 00:00 2024-05-08T17:14:08.889 INFO:teuthology.orchestra.run.smithi154.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 429 kB/s | 172 kB 00:00 2024-05-08T17:14:08.988 INFO:teuthology.orchestra.run.smithi154.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 61 kB/s | 11 kB 00:00 2024-05-08T17:14:09.130 INFO:teuthology.orchestra.run.smithi154.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 73 kB/s | 18 kB 00:00 2024-05-08T17:14:09.231 INFO:teuthology.orchestra.run.smithi154.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 96 kB/s | 23 kB 00:00 2024-05-08T17:14:09.281 INFO:teuthology.orchestra.run.smithi154.stdout:(25/36): python3-cherrypy-18.6.1-2.el9.noarch.r 670 kB/s | 358 kB 00:00 2024-05-08T17:14:09.373 INFO:teuthology.orchestra.run.smithi154.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 80 kB/s | 19 kB 00:00 2024-05-08T17:14:09.523 INFO:teuthology.orchestra.run.smithi154.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 67 kB/s | 20 kB 00:00 2024-05-08T17:14:09.615 INFO:teuthology.orchestra.run.smithi154.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 139 kB/s | 46 kB 00:00 2024-05-08T17:14:09.765 INFO:teuthology.orchestra.run.smithi154.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 201 kB/s | 79 kB 00:00 2024-05-08T17:14:09.857 INFO:teuthology.orchestra.run.smithi154.stdout:(30/36): python3-portend-3.1.0-2.el9.noarch.rpm 68 kB/s | 16 kB 00:00 2024-05-08T17:14:10.008 INFO:teuthology.orchestra.run.smithi154.stdout:(31/36): python3-pecan-1.4.2-3.el9.noarch.rpm 563 kB/s | 272 kB 00:00 2024-05-08T17:14:10.108 INFO:teuthology.orchestra.run.smithi154.stdout:(32/36): python3-tempora-5.0.0-2.el9.noarch.rpm 143 kB/s | 36 kB 00:00 2024-05-08T17:14:10.175 INFO:teuthology.orchestra.run.smithi154.stdout:(33/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 220 kB/s | 90 kB 00:00 2024-05-08T17:14:10.425 INFO:teuthology.orchestra.run.smithi154.stdout:(34/36): python3-zc-lockfile-2.0-10.el9.noarch. 80 kB/s | 20 kB 00:00 2024-05-08T17:14:10.492 INFO:teuthology.orchestra.run.smithi154.stdout:(35/36): python3-webob-1.8.7-6.el9.noarch.rpm 474 kB/s | 230 kB 00:00 2024-05-08T17:14:10.567 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/4 2024-05-08T17:14:10.567 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/4 2024-05-08T17:14:10.567 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/4 2024-05-08T17:14:10.568 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-05-08T17:14:10.667 INFO:teuthology.orchestra.run.smithi154.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 764 kB/s | 427 kB 00:00 2024-05-08T17:14:10.670 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:10.671 INFO:teuthology.orchestra.run.smithi154.stdout:Total 5.6 MB/s | 30 MB 00:05 2024-05-08T17:14:10.815 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:14:10.855 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:14:10.855 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-test-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout: libcephsqlite-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:10.859 INFO:teuthology.orchestra.run.smithi183.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T17:14:10.860 INFO:teuthology.orchestra.run.smithi183.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T17:14:10.860 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:10.860 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:11.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph 2024-05-08T17:14:11.306 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:14:11.307 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:14:11.687 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:41 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:11.818 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:11.820 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:11.820 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:11.820 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 6.6 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 2.4 M 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 1.5 M 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 249 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 4.7 M 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 17 M 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T17:14:11.821 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T17:14:11.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:11.823 INFO:teuthology.orchestra.run.smithi183.stdout:Install 36 Packages 2024-05-08T17:14:11.824 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:11.825 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 30 M 2024-05-08T17:14:11.825 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 107 M 2024-05-08T17:14:11.825 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:12.551 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:14:12.603 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T17:14:12.628 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T17:14:12.664 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T17:14:12.737 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T17:14:12.780 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T17:14:12.808 INFO:teuthology.orchestra.run.smithi183.stdout:(1/36): ceph-19.0.0-2517.g95a4ad3c.el9.x86_64.r 26 kB/s | 6.6 kB 00:00 2024-05-08T17:14:12.812 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T17:14:12.906 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T17:14:12.967 INFO:teuthology.orchestra.run.smithi183.stdout:(2/36): ceph-mds-19.0.0-2517.g95a4ad3c.el9.x86_ 5.8 MB/s | 2.4 MB 00:00 2024-05-08T17:14:12.992 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T17:14:13.009 INFO:teuthology.orchestra.run.smithi183.stdout:(3/36): ceph-mgr-19.0.0-2517.g95a4ad3c.el9.x86_ 3.3 MB/s | 1.5 MB 00:00 2024-05-08T17:14:13.041 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T17:14:13.075 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T17:14:13.105 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T17:14:13.131 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T17:14:13.134 INFO:teuthology.orchestra.run.smithi183.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2517.g95a4 1.9 MB/s | 249 kB 00:00 2024-05-08T17:14:13.165 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T17:14:13.206 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T17:14:13.241 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T17:14:13.275 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T17:14:13.322 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T17:14:13.396 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T17:14:13.536 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T17:14:13.623 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T17:14:13.675 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T17:14:13.714 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T17:14:13.734 INFO:teuthology.orchestra.run.smithi183.stdout:(5/36): ceph-osd-19.0.0-2517.g95a4ad3c.el9.x86_ 22 MB/s | 17 MB 00:00 2024-05-08T17:14:13.759 INFO:teuthology.orchestra.run.smithi183.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 405 kB/s | 253 kB 00:00 2024-05-08T17:14:13.771 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T17:14:13.832 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T17:14:13.868 INFO:teuthology.orchestra.run.smithi183.stdout:(7/36): python3-ply-3.11-14.el9.noarch.rpm 982 kB/s | 106 kB 00:00 2024-05-08T17:14:14.002 INFO:teuthology.orchestra.run.smithi183.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.0 MB/s | 135 kB 00:00 2024-05-08T17:14:14.030 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T17:14:14.076 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T17:14:14.119 INFO:teuthology.orchestra.run.smithi183.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 1.0 MB/s | 125 kB 00:00 2024-05-08T17:14:14.155 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T17:14:14.243 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T17:14:14.252 INFO:teuthology.orchestra.run.smithi183.stdout:(10/36): python3-urllib3-1.26.5-5.el9.noarch.rp 1.6 MB/s | 215 kB 00:00 2024-05-08T17:14:14.283 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T17:14:14.408 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T17:14:14.427 INFO:teuthology.orchestra.run.smithi183.stdout:(11/36): lua-5.4.4-4.el9.x86_64.rpm 1.1 MB/s | 188 kB 00:00 2024-05-08T17:14:14.486 INFO:teuthology.orchestra.run.smithi183.stdout:(12/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T17:14:14.564 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 31/36 2024-05-08T17:14:14.578 INFO:teuthology.orchestra.run.smithi183.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 380 kB/s | 35 kB 00:00 2024-05-08T17:14:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T17:14:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T17:14:14.619 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:14.619 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T17:14:14.619 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T17:14:14.619 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T17:14:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:14.629 INFO:teuthology.orchestra.run.smithi183.stdout:(14/36): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-08T17:14:14.679 INFO:teuthology.orchestra.run.smithi183.stdout:(15/36): python3-toml-0.10.2-6.el9.noarch.rpm 835 kB/s | 42 kB 00:00 2024-05-08T17:14:14.729 INFO:teuthology.orchestra.run.smithi183.stdout:(16/36): python3-cryptography-36.0.1-4.el9.x86_ 1.2 MB/s | 1.2 MB 00:00 2024-05-08T17:14:14.863 INFO:teuthology.orchestra.run.smithi183.stdout:(17/36): lua-devel-5.4.4-4.el9.x86_64.rpm 121 kB/s | 22 kB 00:00 2024-05-08T17:14:15.121 INFO:teuthology.orchestra.run.smithi183.stdout:(18/36): luarocks-3.9.2-1.el9.noarch.rpm 385 kB/s | 151 kB 00:00 2024-05-08T17:14:15.138 INFO:teuthology.orchestra.run.smithi183.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 158 kB/s | 43 kB 00:00 2024-05-08T17:14:15.238 INFO:teuthology.orchestra.run.smithi183.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-05-08T17:14:15.339 INFO:teuthology.orchestra.run.smithi183.stdout:(21/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-05-08T17:14:15.356 INFO:teuthology.orchestra.run.smithi183.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 92 kB/s | 11 kB 00:00 2024-05-08T17:14:15.397 INFO:teuthology.orchestra.run.smithi183.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 427 kB/s | 18 kB 00:00 2024-05-08T17:14:15.414 INFO:teuthology.orchestra.run.smithi183.stdout:(24/36): python3-jaraco-collections-3.0.0-8.el9 397 kB/s | 23 kB 00:00 2024-05-08T17:14:15.440 INFO:teuthology.orchestra.run.smithi183.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 466 kB/s | 19 kB 00:00 2024-05-08T17:14:15.457 INFO:teuthology.orchestra.run.smithi183.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 471 kB/s | 20 kB 00:00 2024-05-08T17:14:15.482 INFO:teuthology.orchestra.run.smithi183.stdout:(27/36): python3-logutils-0.3.5-21.el9.noarch.r 1.1 MB/s | 46 kB 00:00 2024-05-08T17:14:15.550 INFO:teuthology.orchestra.run.smithi183.stdout:(28/36): ceph-mon-19.0.0-2517.g95a4ad3c.el9.x86 1.7 MB/s | 4.7 MB 00:02 2024-05-08T17:14:15.574 INFO:teuthology.orchestra.run.smithi183.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 673 kB/s | 79 kB 00:00 2024-05-08T17:14:15.599 INFO:teuthology.orchestra.run.smithi183.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 2.3 MB/s | 272 kB 00:00 2024-05-08T17:14:15.625 INFO:teuthology.orchestra.run.smithi183.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 219 kB/s | 16 kB 00:00 2024-05-08T17:14:15.650 INFO:teuthology.orchestra.run.smithi183.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-05-08T17:14:15.675 INFO:teuthology.orchestra.run.smithi183.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 477 kB/s | 36 kB 00:00 2024-05-08T17:14:15.717 INFO:teuthology.orchestra.run.smithi183.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 3.4 MB/s | 230 kB 00:00 2024-05-08T17:14:15.826 INFO:teuthology.orchestra.run.smithi183.stdout:(35/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.4 MB/s | 427 kB 00:00 2024-05-08T17:14:15.851 INFO:teuthology.orchestra.run.smithi183.stdout:(36/36): python3-zc-lockfile-2.0-10.el9.noarch. 114 kB/s | 20 kB 00:00 2024-05-08T17:14:15.854 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:15.854 INFO:teuthology.orchestra.run.smithi183.stdout:Total 7.5 MB/s | 30 MB 00:04 2024-05-08T17:14:15.997 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:16.037 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:16.037 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:16.114 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T17:14:16.144 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:16.466 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:16.466 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:16.599 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T17:14:16.624 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:16.858 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T17:14:16.882 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T17:14:16.883 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:16.883 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T17:14:16.883 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T17:14:16.883 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T17:14:16.883 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:16.956 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T17:14:17.729 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:17.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-05-08T17:14:17.819 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-05-08T17:14:17.848 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-05-08T17:14:17.918 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-05-08T17:14:17.971 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-05-08T17:14:18.004 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-05-08T17:14:18.095 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-05-08T17:14:18.191 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-05-08T17:14:18.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-05-08T17:14:18.266 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-05-08T17:14:18.297 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-05-08T17:14:18.331 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-05-08T17:14:18.364 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-05-08T17:14:18.405 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-05-08T17:14:18.440 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-05-08T17:14:18.475 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-05-08T17:14:18.514 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-05-08T17:14:18.585 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/36 2024-05-08T17:14:18.720 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 6/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T17:14:18.721 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T17:14:18.722 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T17:14:18.722 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T17:14:18.722 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T17:14:18.722 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T17:14:18.722 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T17:14:18.723 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T17:14:18.724 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T17:14:18.724 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T17:14:18.724 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T17:14:18.724 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-05-08T17:14:18.804 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-05-08T17:14:18.865 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-05-08T17:14:18.904 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-05-08T17:14:18.952 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-05-08T17:14:19.008 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-05-08T17:14:19.199 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-05-08T17:14:19.250 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-05-08T17:14:19.327 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-05-08T17:14:19.417 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-05-08T17:14:19.522 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-05-08T17:14:19.773 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:19.827 INFO:teuthology.orchestra.run.smithi154.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T17:14:19.828 INFO:teuthology.orchestra.run.smithi154.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T17:14:19.829 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:19.830 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:19.930 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 31/36 2024-05-08T17:14:19.959 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T17:14:19.985 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 32/36 2024-05-08T17:14:19.986 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:19.986 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T17:14:19.986 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T17:14:19.986 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T17:14:19.986 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:20.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-base 2024-05-08T17:14:20.723 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:57 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:20.803 INFO:teuthology.orchestra.run.smithi154.stdout:Package ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:14:20.845 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:20.845 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:14:20.846 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:20.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install cephadm 2024-05-08T17:14:21.428 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T17:14:21.454 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 33/36 2024-05-08T17:14:21.454 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:21.454 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T17:14:21.454 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T17:14:21.454 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T17:14:21.455 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:21.474 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:00:58 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:21.598 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:21.598 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:21.598 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:14:21.598 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 763 k 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 763 k 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 769 k 2024-05-08T17:14:21.599 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:14:21.902 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T17:14:21.925 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 34/36 2024-05-08T17:14:21.926 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:21.926 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T17:14:21.926 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T17:14:21.926 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T17:14:21.926 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:22.153 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 35/36 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T17:14:22.177 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:22.241 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T17:14:22.642 INFO:teuthology.orchestra.run.smithi154.stdout:cephadm-19.0.0-2517.g95a4ad3c.el9.noarch.rpm 732 kB/s | 763 kB 00:01 2024-05-08T17:14:22.643 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:22.643 INFO:teuthology.orchestra.run.smithi154.stdout:Total 731 kB/s | 763 kB 00:01 2024-05-08T17:14:22.643 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:14:22.648 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:14:22.648 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:14:22.654 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:14:22.655 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:14:22.731 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:14:23.336 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:23.412 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:23.705 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:23.808 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 36/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 3/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 4/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 5/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9. 6/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-05-08T17:14:23.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-05-08T17:14:23.810 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-05-08T17:14:23.811 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-05-08T17:14:23.812 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:23.960 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:24.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mds-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-modules-core-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:24.536 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mon-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-osd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T17:14:24.537 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T17:14:24.538 INFO:teuthology.orchestra.run.smithi183.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T17:14:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T17:14:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T17:14:24.539 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:24.539 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:24.759 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:01 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:24.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-base 2024-05-08T17:14:24.887 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:24.887 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:24.887 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repo Size 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 145 k 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 145 k 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 436 k 2024-05-08T17:14:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:14:25.115 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-immutable-object-cache-19.0.0-2517.g95a4ad 638 kB/s | 145 kB 00:00 2024-05-08T17:14:25.116 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:25.116 INFO:teuthology.orchestra.run.smithi154.stdout:Total 634 kB/s | 145 kB 00:00 2024-05-08T17:14:25.116 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:14:25.124 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:14:25.124 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:14:25.180 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:14:25.180 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:14:25.332 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:14:25.391 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:25.418 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:25.418 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:25.418 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T17:14:25.418 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:25.426 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:55 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:25.508 INFO:teuthology.orchestra.run.smithi183.stdout:Package ceph-base-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:14:25.549 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:25.550 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:14:25.550 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:25.620 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install cephadm 2024-05-08T17:14:26.185 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:56 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:26.309 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:26.309 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 763 k 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 763 k 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 769 k 2024-05-08T17:14:26.310 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:26.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:26.361 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:26.361 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:14:26.361 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:26.361 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:26.361 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:26.577 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr 2024-05-08T17:14:27.153 INFO:teuthology.orchestra.run.smithi183.stdout:cephadm-19.0.0-2517.g95a4ad3c.el9.noarch.rpm 908 kB/s | 763 kB 00:00 2024-05-08T17:14:27.153 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:27.153 INFO:teuthology.orchestra.run.smithi183.stdout:Total 906 kB/s | 763 kB 00:00 2024-05-08T17:14:27.153 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:27.153 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:27.158 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:27.158 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:27.164 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:27.164 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:27.239 INFO:teuthology.orchestra.run.smithi154.stdout:Package ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:14:27.240 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:27.281 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:27.330 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:14:27.331 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:27.358 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T17:14:27.695 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:27.772 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:27.921 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:04 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:28.044 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:28.045 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:28.045 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:14:28.045 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:28.046 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:14:28.046 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 3.5 M 2024-05-08T17:14:28.046 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:14:28.046 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T17:14:28.046 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 32 k 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 16 k 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T17:14:28.047 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:14:28.048 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:14:28.049 INFO:teuthology.orchestra.run.smithi154.stdout:Install 12 Packages 2024-05-08T17:14:28.049 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:28.050 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 8.6 M 2024-05-08T17:14:28.050 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 103 M 2024-05-08T17:14:28.050 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:14:28.177 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:28.427 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/1 2024-05-08T17:14:28.427 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:28.427 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:28.427 INFO:teuthology.orchestra.run.smithi183.stdout: cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:28.428 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:28.428 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:28.639 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-immutable-object-cache 2024-05-08T17:14:28.712 INFO:teuthology.orchestra.run.smithi154.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2517.g95a 83 kB/s | 16 kB 00:00 2024-05-08T17:14:28.746 INFO:teuthology.orchestra.run.smithi154.stdout:(2/12): ceph-grafana-dashboards-19.0.0-2517.g95 137 kB/s | 32 kB 00:00 2024-05-08T17:14:28.971 INFO:teuthology.orchestra.run.smithi154.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.2 MB/s | 269 kB 00:00 2024-05-08T17:14:29.105 INFO:teuthology.orchestra.run.smithi154.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 2.6 MB/s | 1.0 MB 00:00 2024-05-08T17:14:29.212 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:00:59 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:29.264 INFO:teuthology.orchestra.run.smithi154.stdout:(5/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.9 MB/s | 863 kB 00:00 2024-05-08T17:14:29.336 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repo Size 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 145 k 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:29.337 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:29.338 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:14:29.338 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:29.338 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 145 k 2024-05-08T17:14:29.338 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 436 k 2024-05-08T17:14:29.338 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:29.683 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-immutable-object-cache-19.0.0-2517.g95a4ad 420 kB/s | 145 kB 00:00 2024-05-08T17:14:29.683 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:29.684 INFO:teuthology.orchestra.run.smithi183.stdout:Total 418 kB/s | 145 kB 00:00 2024-05-08T17:14:29.684 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:29.691 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:29.691 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:29.714 INFO:teuthology.orchestra.run.smithi154.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 43 kB/s | 19 kB 00:00 2024-05-08T17:14:29.748 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:29.748 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:29.897 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:29.967 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:29.993 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:29.993 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:14:29.993 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T17:14:29.993 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:30.106 INFO:teuthology.orchestra.run.smithi154.stdout:(7/12): ceph-mgr-dashboard-19.0.0-2517.g95a4ad3 2.2 MB/s | 3.5 MB 00:01 2024-05-08T17:14:30.314 INFO:teuthology.orchestra.run.smithi154.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 453 kB/s | 548 kB 00:01 2024-05-08T17:14:30.440 INFO:teuthology.orchestra.run.smithi154.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 246 kB/s | 31 kB 00:00 2024-05-08T17:14:30.732 INFO:teuthology.orchestra.run.smithi154.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 646 kB/s | 188 kB 00:00 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c. 1/1 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-immutable-object-cache-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:30.877 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:30.898 INFO:teuthology.orchestra.run.smithi154.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 182 kB/s | 144 kB 00:00 2024-05-08T17:14:31.081 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-mgr 2024-05-08T17:14:31.655 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:01 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:31.739 INFO:teuthology.orchestra.run.smithi183.stdout:Package ceph-mgr-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:14:31.781 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:31.782 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:14:31.782 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:31.852 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-mgr-dashboard 2024-05-08T17:14:32.409 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:02 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:32.423 INFO:teuthology.orchestra.run.smithi154.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 770 kB/s | 2.0 MB 00:02 2024-05-08T17:14:32.428 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:32.428 INFO:teuthology.orchestra.run.smithi154.stdout:Total 2.0 MB/s | 8.6 MB 00:04 2024-05-08T17:14:32.481 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:14:32.499 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:14:32.499 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:14:32.532 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 3.5 M 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 32 k 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 16 k 2024-05-08T17:14:32.533 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout:Install 12 Packages 2024-05-08T17:14:32.534 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:32.535 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 8.6 M 2024-05-08T17:14:32.535 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 103 M 2024-05-08T17:14:32.535 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:32.674 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:14:32.675 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:14:33.097 INFO:teuthology.orchestra.run.smithi183.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2517.g95 170 kB/s | 32 kB 00:00 2024-05-08T17:14:33.122 INFO:teuthology.orchestra.run.smithi183.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2517.g95a 78 kB/s | 16 kB 00:00 2024-05-08T17:14:33.281 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:14:33.298 INFO:teuthology.orchestra.run.smithi183.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.5 MB/s | 269 kB 00:00 2024-05-08T17:14:33.331 INFO:teuthology.orchestra.run.smithi183.stdout:(4/12): protobuf-3.14.0-13.el9.x86_64.rpm 4.3 MB/s | 1.0 MB 00:00 2024-05-08T17:14:33.360 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T17:14:33.395 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T17:14:33.441 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T17:14:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T17:14:33.548 INFO:teuthology.orchestra.run.smithi183.stdout:(5/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 2.5 MB/s | 548 kB 00:00 2024-05-08T17:14:33.577 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T17:14:33.615 INFO:teuthology.orchestra.run.smithi183.stdout:(6/12): ceph-mgr-dashboard-19.0.0-2517.g95a4ad3 5.0 MB/s | 3.5 MB 00:00 2024-05-08T17:14:33.640 INFO:teuthology.orchestra.run.smithi183.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 211 kB/s | 19 kB 00:00 2024-05-08T17:14:33.691 INFO:teuthology.orchestra.run.smithi183.stdout:(8/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 2.8 MB/s | 144 kB 00:00 2024-05-08T17:14:33.708 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T17:14:33.733 INFO:teuthology.orchestra.run.smithi183.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 739 kB/s | 31 kB 00:00 2024-05-08T17:14:33.791 INFO:teuthology.orchestra.run.smithi183.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.2 MB/s | 188 kB 00:00 2024-05-08T17:14:33.809 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T17:14:33.862 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T17:14:33.889 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T17:14:33.921 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 10/12 2024-05-08T17:14:34.100 INFO:teuthology.orchestra.run.smithi183.stdout:(11/12): protobuf-compiler-3.14.0-13.el9.x86_64 1.1 MB/s | 863 kB 00:00 2024-05-08T17:14:34.200 INFO:teuthology.orchestra.run.smithi183.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 3.5 MB/s | 2.0 MB 00:00 2024-05-08T17:14:34.202 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:34.203 INFO:teuthology.orchestra.run.smithi183.stdout:Total 5.2 MB/s | 8.6 MB 00:01 2024-05-08T17:14:34.252 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:34.269 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:34.269 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:34.441 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:34.441 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:34.969 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 11/12 2024-05-08T17:14:35.004 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T17:14:35.050 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:35.151 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-05-08T17:14:35.186 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-05-08T17:14:35.232 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-05-08T17:14:35.263 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-05-08T17:14:35.366 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-05-08T17:14:35.489 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-05-08T17:14:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-05-08T17:14:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-05-08T17:14:35.654 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-05-08T17:14:35.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 10/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 1/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 2/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 3/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T17:14:36.406 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T17:14:36.407 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T17:14:36.407 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T17:14:36.407 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T17:14:36.722 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 11/12 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:36.724 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:36.725 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:14:36.761 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T17:14:36.963 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T17:14:37.530 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:14 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:14:37.652 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:14:37.653 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-05-08T17:14:37.653 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:14:37.653 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 7.4 M 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T17:14:37.654 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout:======================================================================================= 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout:Install 12 Packages 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 38 M 2024-05-08T17:14:37.655 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 209 M 2024-05-08T17:14:37.656 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:14:37.934 INFO:teuthology.orchestra.run.smithi154.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.0 MB/s | 195 kB 00:00 2024-05-08T17:14:38.051 INFO:teuthology.orchestra.run.smithi154.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.5 MB/s | 803 kB 00:00 2024-05-08T17:14:38.110 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 12/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el 1/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noa 2/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9 3/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-05-08T17:14:38.111 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-grafana-dashboards-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-dashboard-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-prometheus-alerts-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T17:14:38.410 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:38.411 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:38.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-mgr-diskprediction-local 2024-05-08T17:14:39.202 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:09 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:39.326 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 7.4 M 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T17:14:39.328 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout:======================================================================================= 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout:Install 12 Packages 2024-05-08T17:14:39.329 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:39.330 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 38 M 2024-05-08T17:14:39.330 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 209 M 2024-05-08T17:14:39.330 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:39.724 INFO:teuthology.orchestra.run.smithi183.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 629 kB/s | 195 kB 00:00 2024-05-08T17:14:39.907 INFO:teuthology.orchestra.run.smithi183.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 803 kB 00:00 2024-05-08T17:14:39.935 INFO:teuthology.orchestra.run.smithi154.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-25 3.4 MB/s | 7.4 MB 00:02 2024-05-08T17:14:40.174 INFO:teuthology.orchestra.run.smithi183.stdout:(3/12): ceph-mgr-diskprediction-local-19.0.0-25 9.7 MB/s | 7.4 MB 00:00 2024-05-08T17:14:40.474 INFO:teuthology.orchestra.run.smithi183.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 44 kB/s | 33 kB 00:00 2024-05-08T17:14:40.833 INFO:teuthology.orchestra.run.smithi183.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 29 kB/s | 18 kB 00:00 2024-05-08T17:14:41.233 INFO:teuthology.orchestra.run.smithi183.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 48 kB/s | 35 kB 00:00 2024-05-08T17:14:41.283 INFO:teuthology.orchestra.run.smithi183.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 2.2 MB/s | 3.0 MB 00:01 2024-05-08T17:14:42.084 INFO:teuthology.orchestra.run.smithi183.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 287 kB/s | 244 kB 00:00 2024-05-08T17:14:42.184 INFO:teuthology.orchestra.run.smithi183.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 3.5 MB/s | 4.7 MB 00:01 2024-05-08T17:14:42.609 INFO:teuthology.orchestra.run.smithi183.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 3.8 MB/s | 5.1 MB 00:01 2024-05-08T17:14:42.950 INFO:teuthology.orchestra.run.smithi183.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 289 kB/s | 250 kB 00:00 2024-05-08T17:14:43.743 INFO:teuthology.orchestra.run.smithi183.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 10 MB/s | 16 MB 00:01 2024-05-08T17:14:43.744 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:43.744 INFO:teuthology.orchestra.run.smithi183.stdout:Total 8.6 MB/s | 38 MB 00:04 2024-05-08T17:14:44.098 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:44.125 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:44.126 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:44.558 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:44.559 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:45.306 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:45.381 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T17:14:45.432 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T17:14:45.542 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T17:14:45.584 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T17:14:45.762 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T17:14:45.799 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T17:14:45.877 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T17:14:46.537 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T17:14:46.654 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T17:14:47.908 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T17:14:48.936 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T17:14:48.972 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T17:14:51.369 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T17:14:51.369 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 1/12 2024-05-08T17:14:51.369 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T17:14:51.369 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T17:14:51.370 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T17:14:51.767 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T17:14:51.767 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:51.767 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:51.767 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:51.767 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:51.768 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:52.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-mgr-rook 2024-05-08T17:14:52.719 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:22 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:52.842 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:52.843 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:52.843 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 49 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T17:14:52.844 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout:Install 13 Packages 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:52.845 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 2.2 M 2024-05-08T17:14:52.846 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 26 M 2024-05-08T17:14:52.846 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:53.439 INFO:teuthology.orchestra.run.smithi183.stdout:(1/13): ceph-mgr-rook-19.0.0-2517.g95a4ad3c.el9 180 kB/s | 49 kB 00:00 2024-05-08T17:14:53.708 INFO:teuthology.orchestra.run.smithi154.stdout:(4/12): flexiblas-3.0.4-8.el9.x86_64.rpm 2.1 kB/s | 33 kB 00:15 2024-05-08T17:14:53.725 INFO:teuthology.orchestra.run.smithi154.stdout:(5/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 1.3 kB/s | 18 kB 00:13 2024-05-08T17:14:53.767 INFO:teuthology.orchestra.run.smithi154.stdout:(6/12): openblas-0.3.21-2.el9.x86_64.rpm 606 kB/s | 35 kB 00:00 2024-05-08T17:14:53.942 INFO:teuthology.orchestra.run.smithi154.stdout:(7/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 193 kB/s | 3.0 MB 00:15 2024-05-08T17:14:53.992 INFO:teuthology.orchestra.run.smithi154.stdout:(8/12): python3-devel-3.9.18-3.el9.x86_64.rpm 1.1 MB/s | 244 kB 00:00 2024-05-08T17:14:54.284 INFO:teuthology.orchestra.run.smithi154.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 859 kB/s | 250 kB 00:00 2024-05-08T17:14:54.747 INFO:teuthology.orchestra.run.smithi183.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 16 kB/s | 26 kB 00:01 2024-05-08T17:14:54.764 INFO:teuthology.orchestra.run.smithi183.stdout:(3/13): python3-jsonpointer-2.0-4.el9.noarch.rp 12 kB/s | 19 kB 00:01 2024-05-08T17:14:54.793 INFO:teuthology.orchestra.run.smithi154.stdout:(10/12): openblas-openmp-0.3.21-2.el9.x86_64.rp 4.4 MB/s | 4.7 MB 00:01 2024-05-08T17:14:55.056 INFO:teuthology.orchestra.run.smithi183.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 137 kB/s | 222 kB 00:01 2024-05-08T17:14:55.089 INFO:teuthology.orchestra.run.smithi183.stdout:(5/13): python3-pyasn1-modules-0.4.8-6.el9.noar 858 kB/s | 279 kB 00:00 2024-05-08T17:14:55.231 INFO:teuthology.orchestra.run.smithi183.stdout:(6/13): python3-cachetools-4.2.4-1.el9.noarch.r 227 kB/s | 32 kB 00:00 2024-05-08T17:14:55.248 INFO:teuthology.orchestra.run.smithi183.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 279 kB/s | 54 kB 00:00 2024-05-08T17:14:55.273 INFO:teuthology.orchestra.run.smithi183.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 358 kB/s | 15 kB 00:00 2024-05-08T17:14:55.340 INFO:teuthology.orchestra.run.smithi183.stdout:(9/13): python3-google-auth-2.29.0-1.el9.noarch 2.2 MB/s | 206 kB 00:00 2024-05-08T17:14:55.399 INFO:teuthology.orchestra.run.smithi183.stdout:(10/13): python3-kubernetes-26.1.0-2.el9.noarch 8.2 MB/s | 1.0 MB 00:00 2024-05-08T17:14:55.416 INFO:teuthology.orchestra.run.smithi183.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 794 kB/s | 59 kB 00:00 2024-05-08T17:14:55.460 INFO:teuthology.orchestra.run.smithi183.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 1.4 MB/s | 90 kB 00:00 2024-05-08T17:14:55.650 INFO:teuthology.orchestra.run.smithi183.stdout:(13/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 179 kB/s | 159 kB 00:00 2024-05-08T17:14:55.652 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:55.652 INFO:teuthology.orchestra.run.smithi183.stdout:Total 803 kB/s | 2.2 MB 00:02 2024-05-08T17:14:55.724 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:14:55.740 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:14:55.740 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:14:55.826 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:14:55.826 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:14:56.329 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:14:56.493 INFO:teuthology.orchestra.run.smithi154.stdout:(11/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 2.0 MB/s | 5.1 MB 00:02 2024-05-08T17:14:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T17:14:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T17:14:56.614 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T17:14:56.652 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T17:14:56.693 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T17:14:56.787 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T17:14:56.889 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T17:14:56.936 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T17:14:57.653 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T17:14:57.717 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T17:14:57.750 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T17:14:57.798 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T17:14:57.820 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T17:14:58.126 INFO:teuthology.orchestra.run.smithi154.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 4.2 MB/s | 16 MB 00:03 2024-05-08T17:14:58.147 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:14:58.147 INFO:teuthology.orchestra.run.smithi154.stdout:Total 1.9 MB/s | 38 MB 00:20 2024-05-08T17:14:58.486 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:14:58.513 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:14:58.514 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:14:58.558 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T17:14:58.559 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T17:14:58.874 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T17:14:58.875 INFO:teuthology.orchestra.run.smithi183.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T17:14:58.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T17:14:58.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T17:14:58.876 INFO:teuthology.orchestra.run.smithi183.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T17:14:58.876 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:58.876 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:14:58.939 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:14:58.940 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:14:59.130 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T17:14:59.690 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:14:59.698 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:29 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:14:59.782 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-05-08T17:14:59.821 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:14:59.821 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 142 k 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:14:59.822 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:14:59.823 INFO:teuthology.orchestra.run.smithi183.stdout:Install 6 Packages 2024-05-08T17:14:59.823 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:14:59.823 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 7.0 M 2024-05-08T17:14:59.823 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 33 M 2024-05-08T17:14:59.823 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:14:59.832 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-05-08T17:14:59.931 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-05-08T17:14:59.967 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-05-08T17:15:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-05-08T17:15:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-05-08T17:15:00.192 INFO:teuthology.orchestra.run.smithi183.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2517.g95a4ad3c.e 640 kB/s | 142 kB 00:00 2024-05-08T17:15:00.250 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-05-08T17:15:00.426 INFO:teuthology.orchestra.run.smithi183.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.3 MB/s | 548 kB 00:00 2024-05-08T17:15:00.476 INFO:teuthology.orchestra.run.smithi183.stdout:(3/6): python3-natsort-7.1.1-5.el9.noarch.rpm 1.1 MB/s | 58 kB 00:00 2024-05-08T17:15:00.526 INFO:teuthology.orchestra.run.smithi183.stdout:(4/6): python3-typing-extensions-4.4.0-2.el9.no 1.0 MB/s | 51 kB 00:00 2024-05-08T17:15:00.585 INFO:teuthology.orchestra.run.smithi183.stdout:(5/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 403 kB/s | 248 kB 00:00 2024-05-08T17:15:00.898 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-05-08T17:15:01.022 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-05-08T17:15:02.259 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T17:15:02.460 INFO:teuthology.orchestra.run.smithi183.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 2.4 MB/s | 6.0 MB 00:02 2024-05-08T17:15:02.462 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:02.463 INFO:teuthology.orchestra.run.smithi183.stdout:Total 2.6 MB/s | 7.0 MB 00:02 2024-05-08T17:15:02.537 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:02.549 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:02.549 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:02.646 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:02.647 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:02.967 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:03.127 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T17:15:03.170 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T17:15:03.279 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-05-08T17:15:03.308 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T17:15:03.695 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T17:15:03.781 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T17:15:03.844 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T17:15:03.874 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T17:15:04.631 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T17:15:04.632 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/6 2024-05-08T17:15:04.632 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T17:15:04.632 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T17:15:04.632 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T17:15:04.632 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T17:15:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T17:15:04.889 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:04.889 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T17:15:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T17:15:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T17:15:04.890 INFO:teuthology.orchestra.run.smithi183.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T17:15:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T17:15:04.891 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:04.891 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:05.166 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-fuse 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 12/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4a 1/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-05-08T17:15:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-05-08T17:15:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-05-08T17:15:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-05-08T17:15:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-05-08T17:15:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-05-08T17:15:05.736 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:35 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:05.859 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:05.859 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 890 k 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout:Install 2 Packages 2024-05-08T17:15:05.860 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:05.861 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 970 k 2024-05-08T17:15:05.861 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 2.8 M 2024-05-08T17:15:05.861 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T17:15:06.112 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T17:15:06.113 INFO:teuthology.orchestra.run.smithi154.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T17:15:06.113 INFO:teuthology.orchestra.run.smithi154.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T17:15:06.113 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:06.113 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:06.270 INFO:teuthology.orchestra.run.smithi183.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 80 kB 00:00 2024-05-08T17:15:06.470 INFO:teuthology.orchestra.run.smithi183.stdout:(2/2): ceph-fuse-19.0.0-2517.g95a4ad3c.el9.x86_ 2.7 MB/s | 890 kB 00:00 2024-05-08T17:15:06.470 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:06.471 INFO:teuthology.orchestra.run.smithi183.stdout:Total 1.6 MB/s | 970 kB 00:00 2024-05-08T17:15:06.483 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:06.505 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:06.505 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:06.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr-rook 2024-05-08T17:15:06.583 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:06.583 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:06.777 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:06.929 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T17:15:06.994 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T17:15:07.106 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:07.229 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 49 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T17:15:07.231 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout:Install 13 Packages 2024-05-08T17:15:07.232 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:07.233 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 2.2 M 2024-05-08T17:15:07.233 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 26 M 2024-05-08T17:15:07.233 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:07.469 INFO:teuthology.orchestra.run.smithi154.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 382 kB/s | 26 kB 00:00 2024-05-08T17:15:07.494 INFO:teuthology.orchestra.run.smithi154.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 208 kB/s | 19 kB 00:00 2024-05-08T17:15:07.570 INFO:teuthology.orchestra.run.smithi154.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.1 MB/s | 159 kB 00:00 2024-05-08T17:15:07.595 INFO:teuthology.orchestra.run.smithi154.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 2.2 MB/s | 222 kB 00:00 2024-05-08T17:15:07.620 INFO:teuthology.orchestra.run.smithi154.stdout:(5/13): ceph-mgr-rook-19.0.0-2517.g95a4ad3c.el9 226 kB/s | 49 kB 00:00 2024-05-08T17:15:07.671 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T17:15:07.671 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/2 2024-05-08T17:15:07.671 INFO:teuthology.orchestra.run.smithi154.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 713 kB/s | 54 kB 00:00 2024-05-08T17:15:07.696 INFO:teuthology.orchestra.run.smithi154.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 2.2 MB/s | 279 kB 00:00 2024-05-08T17:15:07.771 INFO:teuthology.orchestra.run.smithi154.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 214 kB/s | 32 kB 00:00 2024-05-08T17:15:07.796 INFO:teuthology.orchestra.run.smithi154.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 120 kB/s | 15 kB 00:00 2024-05-08T17:15:07.913 INFO:teuthology.orchestra.run.smithi154.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 946 kB/s | 206 kB 00:00 2024-05-08T17:15:07.916 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T17:15:07.916 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:07.917 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:07.917 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T17:15:07.917 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:07.917 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:07.939 INFO:teuthology.orchestra.run.smithi154.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 415 kB/s | 59 kB 00:00 2024-05-08T17:15:08.031 INFO:teuthology.orchestra.run.smithi154.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 765 kB/s | 90 kB 00:00 2024-05-08T17:15:08.139 INFO:teuthology.orchestra.run.smithi154.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 2.8 MB/s | 1.0 MB 00:00 2024-05-08T17:15:08.140 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:08.140 INFO:teuthology.orchestra.run.smithi154.stdout:Total 2.4 MB/s | 2.2 MB 00:00 2024-05-08T17:15:08.205 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:08.208 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install ceph-volume 2024-05-08T17:15:08.219 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:08.220 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:08.307 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:08.308 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:08.783 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:38 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:08.832 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:08.907 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repository Size 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 264 k 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:Installing dependencies: 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:08.908 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout:Install 2 Packages 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 341 k 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 1.5 M 2024-05-08T17:15:08.909 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-05-08T17:15:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-05-08T17:15:09.104 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-05-08T17:15:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-05-08T17:15:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-05-08T17:15:09.285 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-05-08T17:15:09.307 INFO:teuthology.orchestra.run.smithi183.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 296 kB/s | 77 kB 00:00 2024-05-08T17:15:09.396 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-05-08T17:15:09.451 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-05-08T17:15:09.941 INFO:teuthology.orchestra.run.smithi183.stdout:(2/2): ceph-volume-19.0.0-2517.g95a4ad3c.el9.no 295 kB/s | 264 kB 00:00 2024-05-08T17:15:09.941 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:09.941 INFO:teuthology.orchestra.run.smithi183.stdout:Total 330 kB/s | 341 kB 00:01 2024-05-08T17:15:09.956 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:09.966 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:09.966 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:10.002 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:10.003 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:10.153 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:10.172 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-05-08T17:15:10.224 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-05-08T17:15:10.258 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-05-08T17:15:10.306 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-05-08T17:15:10.336 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T17:15:10.340 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T17:15:10.376 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T17:15:10.392 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T17:15:10.393 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:15:10.393 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T17:15:10.393 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:11.119 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 13/13 2024-05-08T17:15:11.119 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/13 2024-05-08T17:15:11.119 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-05-08T17:15:11.120 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-05-08T17:15:11.220 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/2 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-rook-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T17:15:11.437 INFO:teuthology.orchestra.run.smithi154.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:11.438 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:11.518 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T17:15:11.519 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:11.519 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:11.519 INFO:teuthology.orchestra.run.smithi183.stdout: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:11.519 INFO:teuthology.orchestra.run.smithi183.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T17:15:11.519 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:11.520 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:11.689 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-mgr-cephadm 2024-05-08T17:15:11.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install librados-devel 2024-05-08T17:15:12.264 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:12.388 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:12.388 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:12.388 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 142 k 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout:Install 6 Packages 2024-05-08T17:15:12.389 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:12.390 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 7.0 M 2024-05-08T17:15:12.390 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 33 M 2024-05-08T17:15:12.390 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:12.425 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:42 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:12.547 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:12.548 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:12.548 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repo Size 2024-05-08T17:15:12.548 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:12.548 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 127 k 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 127 k 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 456 k 2024-05-08T17:15:12.549 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:12.775 INFO:teuthology.orchestra.run.smithi183.stdout:librados-devel-19.0.0-2517.g95a4ad3c.el9.x86_64 564 kB/s | 127 kB 00:00 2024-05-08T17:15:12.776 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:12.776 INFO:teuthology.orchestra.run.smithi183.stdout:Total 560 kB/s | 127 kB 00:00 2024-05-08T17:15:12.776 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:12.781 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:12.781 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:12.844 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:12.845 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:12.994 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:13.114 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:13.284 INFO:teuthology.orchestra.run.smithi154.stdout:(1/6): ceph-mgr-cephadm-19.0.0-2517.g95a4ad3c.e 381 kB/s | 142 kB 00:00 2024-05-08T17:15:13.546 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:13.659 INFO:teuthology.orchestra.run.smithi154.stdout:(2/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.4 MB/s | 548 kB 00:00 2024-05-08T17:15:13.734 INFO:teuthology.orchestra.run.smithi154.stdout:(3/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 301 kB/s | 248 kB 00:00 2024-05-08T17:15:13.751 INFO:teuthology.orchestra.run.smithi154.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 629 kB/s | 58 kB 00:00 2024-05-08T17:15:13.809 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:13.809 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:13.809 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:13.809 INFO:teuthology.orchestra.run.smithi183.stdout: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:13.809 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:13.810 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:13.902 INFO:teuthology.orchestra.run.smithi154.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 306 kB/s | 51 kB 00:00 2024-05-08T17:15:14.075 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install libcephfs2 2024-05-08T17:15:14.194 INFO:teuthology.orchestra.run.smithi154.stdout:(6/6): python3-babel-2.9.1-2.el9.noarch.rpm 4.7 MB/s | 6.0 MB 00:01 2024-05-08T17:15:14.195 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:14.195 INFO:teuthology.orchestra.run.smithi154.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-05-08T17:15:14.267 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:14.278 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:14.278 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:14.376 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:14.376 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:14.653 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:44 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:14.681 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:14.734 INFO:teuthology.orchestra.run.smithi183.stdout:Package libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:14.775 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:14.776 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:14.776 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:14.834 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-05-08T17:15:14.858 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install libcephfs-devel 2024-05-08T17:15:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-05-08T17:15:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-05-08T17:15:15.434 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:45 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:15.515 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-05-08T17:15:15.555 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repo Size 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 32 k 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:15.556 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:15.557 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:15:15.557 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:15.557 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 32 k 2024-05-08T17:15:15.557 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 142 k 2024-05-08T17:15:15.557 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:15.586 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-05-08T17:15:15.617 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T17:15:15.736 INFO:teuthology.orchestra.run.smithi183.stdout:libcephfs-devel-19.0.0-2517.g95a4ad3c.el9.x86_6 178 kB/s | 32 kB 00:00 2024-05-08T17:15:15.736 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:15.736 INFO:teuthology.orchestra.run.smithi183.stdout:Total 176 kB/s | 32 kB 00:00 2024-05-08T17:15:15.736 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:15.740 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:15.740 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:15.762 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:15.763 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:15.842 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:15.958 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:16.339 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 6/6 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/6 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-05-08T17:15:16.360 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:16.568 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:16.635 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-05-08T17:15:16.635 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-mgr-cephadm-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:16.636 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:16.744 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install librados2 2024-05-08T17:15:16.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-fuse 2024-05-08T17:15:17.314 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:47 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:17.396 INFO:teuthology.orchestra.run.smithi183.stdout:Package librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:17.437 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:17.438 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:17.438 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:17.503 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:17.507 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install librbd1 2024-05-08T17:15:17.624 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:17.624 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:17.624 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 890 k 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout:Install 2 Packages 2024-05-08T17:15:17.625 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:17.626 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 970 k 2024-05-08T17:15:17.626 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 2.8 M 2024-05-08T17:15:17.626 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:17.888 INFO:teuthology.orchestra.run.smithi154.stdout:(1/2): fuse-2.9.9-15.el9.x86_64.rpm 462 kB/s | 80 kB 00:00 2024-05-08T17:15:18.013 INFO:teuthology.orchestra.run.smithi154.stdout:(2/2): ceph-fuse-19.0.0-2517.g95a4ad3c.el9.x86_ 2.9 MB/s | 890 kB 00:00 2024-05-08T17:15:18.013 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:18.014 INFO:teuthology.orchestra.run.smithi154.stdout:Total 2.4 MB/s | 970 kB 00:00 2024-05-08T17:15:18.025 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:18.047 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:18.047 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:18.077 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:18.125 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:18.126 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:18.159 INFO:teuthology.orchestra.run.smithi183.stdout:Package librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:18.200 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:18.201 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:18.202 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:18.277 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install python3-rados 2024-05-08T17:15:18.327 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:18.465 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-05-08T17:15:18.530 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T17:15:18.848 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:48 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:18.929 INFO:teuthology.orchestra.run.smithi183.stdout:Package python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:18.970 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:18.971 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:18.971 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:19.053 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install python3-rgw 2024-05-08T17:15:19.188 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2/2 2024-05-08T17:15:19.188 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/2 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:19.448 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:19.626 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:49 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:19.707 INFO:teuthology.orchestra.run.smithi183.stdout:Package python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:19.749 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:19.750 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:19.750 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:19.768 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install ceph-volume 2024-05-08T17:15:19.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install python3-cephfs 2024-05-08T17:15:20.355 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:01:57 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:20.437 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:50 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:20.484 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repository Size 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume noarch 2:19.0.0-2517.g95a4ad3c.el9 ceph-noarch 264 k 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:Installing dependencies: 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:20.485 INFO:teuthology.orchestra.run.smithi154.stdout:Install 2 Packages 2024-05-08T17:15:20.486 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:20.486 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 341 k 2024-05-08T17:15:20.486 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 1.5 M 2024-05-08T17:15:20.486 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:20.522 INFO:teuthology.orchestra.run.smithi183.stdout:Package python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:20.565 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:20.566 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:20.566 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:20.602 INFO:teuthology.orchestra.run.smithi154.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 1.0 MB/s | 77 kB 00:00 2024-05-08T17:15:20.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install python3-rbd 2024-05-08T17:15:20.769 INFO:teuthology.orchestra.run.smithi154.stdout:(2/2): ceph-volume-19.0.0-2517.g95a4ad3c.el9.no 1.1 MB/s | 264 kB 00:00 2024-05-08T17:15:20.770 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:20.770 INFO:teuthology.orchestra.run.smithi154.stdout:Total 1.2 MB/s | 341 kB 00:00 2024-05-08T17:15:20.781 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:20.791 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:20.791 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:20.826 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:20.826 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:21.154 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-05-08T17:15:21.188 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T17:15:21.204 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2/2 2024-05-08T17:15:21.204 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:15:21.204 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T17:15:21.205 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:21.235 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:51 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:21.315 INFO:teuthology.orchestra.run.smithi183.stdout:Package python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:21.357 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:21.359 INFO:teuthology.orchestra.run.smithi183.stdout:Nothing to do. 2024-05-08T17:15:21.359 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:21.430 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install rbd-fuse 2024-05-08T17:15:21.926 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 1/2 2024-05-08T17:15:22.002 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:52 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:22.124 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 85 k 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 85 k 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 231 k 2024-05-08T17:15:22.125 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout: ceph-volume-2:19.0.0-2517.g95a4ad3c.el9.noarch 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:22.218 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:22.356 INFO:teuthology.orchestra.run.smithi183.stdout:rbd-fuse-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 371 kB/s | 85 kB 00:00 2024-05-08T17:15:22.356 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:22.356 INFO:teuthology.orchestra.run.smithi183.stdout:Total 369 kB/s | 85 kB 00:00 2024-05-08T17:15:22.357 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:22.363 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:22.363 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:22.426 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:22.427 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:22.491 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librados-devel 2024-05-08T17:15:22.570 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:22.686 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:23.071 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:00 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:23.077 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:23.193 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:23.193 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:23.193 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repo Size 2024-05-08T17:15:23.193 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:23.193 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 127 k 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 127 k 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 456 k 2024-05-08T17:15:23.194 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:23.318 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:23.319 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:23.319 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:23.319 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:23.515 INFO:teuthology.orchestra.run.smithi154.stdout:librados-devel-19.0.0-2517.g95a4ad3c.el9.x86_64 395 kB/s | 127 kB 00:00 2024-05-08T17:15:23.516 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:23.516 INFO:teuthology.orchestra.run.smithi154.stdout:Total 394 kB/s | 127 kB 00:00 2024-05-08T17:15:23.516 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:23.522 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:23.522 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:23.565 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install rbd-mirror 2024-05-08T17:15:23.585 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:23.585 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:23.737 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:23.838 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:24.147 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:54 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:24.272 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout: Package Arch Version Repo Size 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T17:15:24.273 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 3.1 M 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 13 M 2024-05-08T17:15:24.274 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:24.278 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:24.523 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: librados-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:24.524 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:24.650 INFO:teuthology.orchestra.run.smithi183.stdout:rbd-mirror-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 8.2 MB/s | 3.1 MB 00:00 2024-05-08T17:15:24.651 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:24.651 INFO:teuthology.orchestra.run.smithi183.stdout:Total 8.2 MB/s | 3.1 MB 00:00 2024-05-08T17:15:24.652 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:24.661 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:24.661 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:24.741 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:24.742 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:24.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install libcephfs2 2024-05-08T17:15:25.168 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:25.244 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:25.268 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:25.268 INFO:teuthology.orchestra.run.smithi183.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:15:25.268 INFO:teuthology.orchestra.run.smithi183.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T17:15:25.268 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T17:15:25.268 INFO:teuthology.orchestra.run.smithi183.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T17:15:25.269 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:25.369 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:02 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:25.451 INFO:teuthology.orchestra.run.smithi154.stdout:Package libcephfs2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:25.493 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:25.494 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:25.494 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:25.580 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install libcephfs-devel 2024-05-08T17:15:26.164 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:03 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:26.286 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:26.287 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:26.287 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repo Size 2024-05-08T17:15:26.287 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 32 k 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 32 k 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 142 k 2024-05-08T17:15:26.288 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:26.336 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:26.544 INFO:teuthology.orchestra.run.smithi154.stdout:libcephfs-devel-19.0.0-2517.g95a4ad3c.el9.x86_6 124 kB/s | 32 kB 00:00 2024-05-08T17:15:26.551 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:26.551 INFO:teuthology.orchestra.run.smithi154.stdout:Total 123 kB/s | 32 kB 00:00 2024-05-08T17:15:26.551 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:26.551 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:26.551 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:26.563 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:26.563 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:26.574 DEBUG:teuthology.orchestra.run.smithi183:> sudo yum -y install rbd-nbd 2024-05-08T17:15:26.642 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:26.759 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:27.156 INFO:teuthology.orchestra.run.smithi183.stdout:Last metadata expiration check: 0:01:57 ago on Wed 08 May 2024 05:13:30 PM UTC. 2024-05-08T17:15:27.282 INFO:teuthology.orchestra.run.smithi183.stdout:Dependencies resolved. 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout: Package Architecture Version Repository Size 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:Installing: 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 170 k 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction Summary 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:================================================================================ 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout:Install 1 Package 2024-05-08T17:15:27.283 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:27.284 INFO:teuthology.orchestra.run.smithi183.stdout:Total download size: 170 k 2024-05-08T17:15:27.284 INFO:teuthology.orchestra.run.smithi183.stdout:Installed size: 494 k 2024-05-08T17:15:27.284 INFO:teuthology.orchestra.run.smithi183.stdout:Downloading Packages: 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout: libcephfs-devel-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:27.365 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:27.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librados2 2024-05-08T17:15:27.637 INFO:teuthology.orchestra.run.smithi183.stdout:rbd-nbd-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 482 kB/s | 170 kB 00:00 2024-05-08T17:15:27.638 INFO:teuthology.orchestra.run.smithi183.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:27.638 INFO:teuthology.orchestra.run.smithi183.stdout:Total 480 kB/s | 170 kB 00:00 2024-05-08T17:15:27.638 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction check 2024-05-08T17:15:27.646 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction check succeeded. 2024-05-08T17:15:27.647 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction test 2024-05-08T17:15:27.711 INFO:teuthology.orchestra.run.smithi183.stdout:Transaction test succeeded. 2024-05-08T17:15:27.711 INFO:teuthology.orchestra.run.smithi183.stdout:Running transaction 2024-05-08T17:15:27.863 INFO:teuthology.orchestra.run.smithi183.stdout: Preparing : 1/1 2024-05-08T17:15:27.977 INFO:teuthology.orchestra.run.smithi183.stdout: Installing : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:28.106 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:05 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:28.189 INFO:teuthology.orchestra.run.smithi154.stdout:Package librados2-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:28.229 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:28.230 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:28.230 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:28.303 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install librbd1 2024-05-08T17:15:28.364 INFO:teuthology.orchestra.run.smithi183.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:28.610 INFO:teuthology.orchestra.run.smithi183.stdout: Verifying : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:28.611 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:28.611 INFO:teuthology.orchestra.run.smithi183.stdout:Installed: 2024-05-08T17:15:28.611 INFO:teuthology.orchestra.run.smithi183.stdout: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:28.611 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:28.611 INFO:teuthology.orchestra.run.smithi183.stdout:Complete! 2024-05-08T17:15:28.875 DEBUG:teuthology.parallel:result is None 2024-05-08T17:15:28.876 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:05 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:28.956 INFO:teuthology.orchestra.run.smithi154.stdout:Package librbd1-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:28.998 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:28.999 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:28.999 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:29.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rados 2024-05-08T17:15:29.654 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:06 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:29.734 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rados-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:29.775 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:29.776 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:29.776 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:29.855 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rgw 2024-05-08T17:15:30.430 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:07 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:30.511 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rgw-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:30.552 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:30.553 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:30.553 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:30.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-cephfs 2024-05-08T17:15:31.200 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:08 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:31.279 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-cephfs-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:31.320 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:31.321 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:31.321 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:31.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install python3-rbd 2024-05-08T17:15:31.967 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:08 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:32.054 INFO:teuthology.orchestra.run.smithi154.stdout:Package python3-rbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 is already installed. 2024-05-08T17:15:32.095 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:32.096 INFO:teuthology.orchestra.run.smithi154.stdout:Nothing to do. 2024-05-08T17:15:32.096 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:32.171 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-fuse 2024-05-08T17:15:32.742 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:09 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:32.865 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 85 k 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:32.866 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:15:32.867 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:32.867 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 85 k 2024-05-08T17:15:32.867 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 231 k 2024-05-08T17:15:32.867 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:33.256 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-fuse-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 219 kB/s | 85 kB 00:00 2024-05-08T17:15:33.256 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:33.256 INFO:teuthology.orchestra.run.smithi154.stdout:Total 218 kB/s | 85 kB 00:00 2024-05-08T17:15:33.257 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:33.264 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:33.264 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:33.328 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:33.329 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:33.571 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:33.961 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-fuse-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:34.222 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:34.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-mirror 2024-05-08T17:15:35.098 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:12 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:35.231 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout: Package Arch Version Repo Size 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 3.1 M 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:35.232 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:15:35.233 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 3.1 M 2024-05-08T17:15:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 13 M 2024-05-08T17:15:35.233 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:35.635 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-mirror-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 7.7 MB/s | 3.1 MB 00:00 2024-05-08T17:15:35.635 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:35.636 INFO:teuthology.orchestra.run.smithi154.stdout:Total 7.7 MB/s | 3.1 MB 00:00 2024-05-08T17:15:35.636 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:35.645 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:35.645 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:35.726 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:35.727 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:36.154 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:36.219 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T17:15:36.245 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-mirror-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:37.441 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:37.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo yum -y install rbd-nbd 2024-05-08T17:15:38.264 INFO:teuthology.orchestra.run.smithi154.stdout:Last metadata expiration check: 0:02:15 ago on Wed 08 May 2024 05:13:23 PM UTC. 2024-05-08T17:15:38.386 INFO:teuthology.orchestra.run.smithi154.stdout:Dependencies resolved. 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout: Package Architecture Version Repository Size 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Installing: 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd x86_64 2:19.0.0-2517.g95a4ad3c.el9 ceph 170 k 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction Summary 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:================================================================================ 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Install 1 Package 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Total download size: 170 k 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Installed size: 494 k 2024-05-08T17:15:38.387 INFO:teuthology.orchestra.run.smithi154.stdout:Downloading Packages: 2024-05-08T17:15:38.730 INFO:teuthology.orchestra.run.smithi154.stdout:rbd-nbd-19.0.0-2517.g95a4ad3c.el9.x86_64.rpm 498 kB/s | 170 kB 00:00 2024-05-08T17:15:38.731 INFO:teuthology.orchestra.run.smithi154.stdout:-------------------------------------------------------------------------------- 2024-05-08T17:15:38.731 INFO:teuthology.orchestra.run.smithi154.stdout:Total 496 kB/s | 170 kB 00:00 2024-05-08T17:15:38.731 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction check 2024-05-08T17:15:38.739 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction check succeeded. 2024-05-08T17:15:38.739 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction test 2024-05-08T17:15:38.802 INFO:teuthology.orchestra.run.smithi154.stdout:Transaction test succeeded. 2024-05-08T17:15:38.803 INFO:teuthology.orchestra.run.smithi154.stdout:Running transaction 2024-05-08T17:15:38.954 INFO:teuthology.orchestra.run.smithi154.stdout: Preparing : 1/1 2024-05-08T17:15:39.069 INFO:teuthology.orchestra.run.smithi154.stdout: Installing : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:39.492 INFO:teuthology.orchestra.run.smithi154.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:39.772 INFO:teuthology.orchestra.run.smithi154.stdout: Verifying : rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 1/1 2024-05-08T17:15:39.772 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:39.772 INFO:teuthology.orchestra.run.smithi154.stdout:Installed: 2024-05-08T17:15:39.772 INFO:teuthology.orchestra.run.smithi154.stdout: rbd-nbd-2:19.0.0-2517.g95a4ad3c.el9.x86_64 2024-05-08T17:15:39.773 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:39.773 INFO:teuthology.orchestra.run.smithi154.stdout:Complete! 2024-05-08T17:15:40.026 DEBUG:teuthology.parallel:result is None 2024-05-08T17:15:40.027 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T17:15:40.249 DEBUG:teuthology.orchestra.run.smithi154:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T17:15:40.271 INFO:teuthology.orchestra.run.smithi154.stdout:19.0.0-2517.g95a4ad3c.el9 2024-05-08T17:15:40.271 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2517.g95a4ad3c.el9 2024-05-08T17:15:40.271 INFO:teuthology.task.install:The correct ceph version 19.0.0-2517.g95a4ad3c is installed. 2024-05-08T17:15:40.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=95a4ad3c90f2f12f920e87725ad8d9002f948722 2024-05-08T17:15:40.501 DEBUG:teuthology.orchestra.run.smithi183:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T17:15:40.527 INFO:teuthology.orchestra.run.smithi183.stdout:19.0.0-2517.g95a4ad3c.el9 2024-05-08T17:15:40.527 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2517.g95a4ad3c.el9 2024-05-08T17:15:40.527 INFO:teuthology.task.install:The correct ceph version 19.0.0-2517.g95a4ad3c is installed. 2024-05-08T17:15:40.529 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T17:15:40.529 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:40.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T17:15:40.555 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:40.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T17:15:40.598 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T17:15:40.598 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:40.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T17:15:40.622 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T17:15:40.687 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:40.687 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T17:15:40.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T17:15:40.782 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T17:15:40.782 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:40.782 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T17:15:40.806 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T17:15:40.871 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:40.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T17:15:40.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T17:15:40.963 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T17:15:40.964 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:40.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T17:15:40.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T17:15:41.055 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:41.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T17:15:41.080 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T17:15:41.145 INFO:teuthology.run_tasks:Running task ceph... 2024-05-08T17:15:41.237 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-05-08T17:15:41.237 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 777 /var/log/ceph 2024-05-08T17:15:41.240 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 777 /var/log/ceph 2024-05-08T17:15:41.264 INFO:tasks.ceph:Disabling ceph logrotate... 2024-05-08T17:15:41.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T17:15:41.305 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/logrotate.d/ceph 2024-05-08T17:15:41.329 INFO:tasks.ceph:Creating extra log directories... 2024-05-08T17:15:41.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T17:15:41.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-05-08T17:15:41.406 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-05-08T17:15:41.407 INFO:tasks.ceph:config {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-05-08T17:15:41.407 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758', 'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'description': 'rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep}', 'email': None, 'first_in_suite': False, 'job_id': '7697758', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 10}}, {'volumes': {'count': 4, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid'}, 'ceph': {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}}, 'owner': 'scheduled_lflores@teuthology', 'priority': 75, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'node-exporter.a'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 8082, 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'sleep_before_teardown': 0, 'subset': '111/120000', 'suite': 'rados', 'suite_branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_95a4ad3c90f2f12f920e87725ad8d9002f948722/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722', 'targets': {'smithi154.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOJrT+ufi9mx4WgjWMF+gLBS30zM9CR4KA5V8u5nsFkF+SR6QfsZY+x96tiHJsKRFWoSo1jbwYdHkLpl8OxhbPk=', 'smithi183.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL0wuaKY8ZS4W3t6jNFd7CeZ9saC5tZuFO8Ko86e3r1b5g9YwFdiC3zamVDAWWV3Y6ZAs8FYVONJpcCYg2e7Q6U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': None}, {'thrashosds': {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200}}, {'mon_recovery': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '1ae7ad82388e92a475afff437d49054826c019a1', 'timestamp': '2024-05-08_14:59:36', 'tube': 'smithi', 'user': 'lflores', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-05-08T17:15:41.407 DEBUG:teuthology.orchestra.run.smithi154:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T17:15:41.435 DEBUG:teuthology.orchestra.run.smithi183:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-05-08T17:15:41.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T17:15:41.491 DEBUG:teuthology.orchestra.run.smithi183:> sudo install -d -m0777 -- /var/run/ceph 2024-05-08T17:15:41.515 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:41.515 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T17:15:41.569 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T17:15:41.569 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_1 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-05-08 17:15:36.495792430 +0000 2024-05-08T17:15:41.626 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-05-08 17:12:42.563211560 +0000 2024-05-08T17:15:41.627 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-05-08 17:12:42.563211560 +0000 2024-05-08T17:15:41.627 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-05-08 17:12:42.563211560 +0000 2024-05-08T17:15:41.628 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T17:15:41.690 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-05-08T17:15:41.690 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-05-08T17:15:41.690 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000204008 s, 2.5 MB/s 2024-05-08T17:15:41.691 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T17:15:41.748 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_2 2024-05-08T17:15:41.803 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-05-08 17:15:36.495792430 +0000 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-05-08 17:12:42.897202325 +0000 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-05-08 17:12:42.897202325 +0000 2024-05-08T17:15:41.804 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-05-08 17:12:42.897202325 +0000 2024-05-08T17:15:41.804 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T17:15:41.866 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-05-08T17:15:41.867 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-05-08T17:15:41.867 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000218715 s, 2.3 MB/s 2024-05-08T17:15:41.868 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T17:15:41.925 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_3 2024-05-08T17:15:41.979 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T17:15:41.979 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-05-08 17:15:36.496792404 +0000 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-05-08 17:12:43.244192729 +0000 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-05-08 17:12:43.244192729 +0000 2024-05-08T17:15:41.980 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-05-08 17:12:43.244192729 +0000 2024-05-08T17:15:41.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T17:15:42.042 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-05-08T17:15:42.042 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-05-08T17:15:42.043 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000167827 s, 3.1 MB/s 2024-05-08T17:15:42.044 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T17:15:42.100 DEBUG:teuthology.orchestra.run.smithi154:> stat /dev/vg_nvme/lv_4 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Access: 2024-05-08 17:15:36.496792404 +0000 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Modify: 2024-05-08 17:12:43.612182553 +0000 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout:Change: 2024-05-08 17:12:43.612182553 +0000 2024-05-08T17:15:42.155 INFO:teuthology.orchestra.run.smithi154.stdout: Birth: 2024-05-08 17:12:43.612182553 +0000 2024-05-08T17:15:42.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T17:15:42.218 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records in 2024-05-08T17:15:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:1+0 records out 2024-05-08T17:15:42.219 INFO:teuthology.orchestra.run.smithi154.stderr:512 bytes copied, 0.000163976 s, 3.1 MB/s 2024-05-08T17:15:42.220 DEBUG:teuthology.orchestra.run.smithi154:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T17:15:42.277 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:42.277 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:42.277 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T17:15:42.294 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T17:15:42.294 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_1 2024-05-08T17:15:42.350 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-08 17:15:25.516997288 +0000 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-08 17:12:42.822708312 +0000 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-08 17:12:42.822708312 +0000 2024-05-08T17:15:42.351 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-08 17:12:42.822708312 +0000 2024-05-08T17:15:42.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T17:15:42.416 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-08T17:15:42.416 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-08T17:15:42.416 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000273174 s, 1.9 MB/s 2024-05-08T17:15:42.417 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T17:15:42.475 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_2 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 687 Links: 1 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-08 17:15:25.517997261 +0000 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-08 17:12:43.175699010 +0000 2024-05-08T17:15:42.531 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-08 17:12:43.175699010 +0000 2024-05-08T17:15:42.532 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-08 17:12:43.175699010 +0000 2024-05-08T17:15:42.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T17:15:42.595 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-08T17:15:42.595 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-08T17:15:42.595 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000171487 s, 3.0 MB/s 2024-05-08T17:15:42.596 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T17:15:42.653 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_3 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-08 17:15:25.517997261 +0000 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-08 17:12:43.522689865 +0000 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-08 17:12:43.522689865 +0000 2024-05-08T17:15:42.713 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-08 17:12:43.522689865 +0000 2024-05-08T17:15:42.714 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T17:15:42.776 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-08T17:15:42.776 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-08T17:15:42.776 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000203665 s, 2.5 MB/s 2024-05-08T17:15:42.777 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T17:15:42.834 DEBUG:teuthology.orchestra.run.smithi183:> stat /dev/vg_nvme/lv_4 2024-05-08T17:15:42.889 INFO:teuthology.orchestra.run.smithi183.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T17:15:42.889 INFO:teuthology.orchestra.run.smithi183.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T17:15:42.889 INFO:teuthology.orchestra.run.smithi183.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T17:15:42.889 INFO:teuthology.orchestra.run.smithi183.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T17:15:42.890 INFO:teuthology.orchestra.run.smithi183.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T17:15:42.890 INFO:teuthology.orchestra.run.smithi183.stdout:Access: 2024-05-08 17:15:25.517997261 +0000 2024-05-08T17:15:42.890 INFO:teuthology.orchestra.run.smithi183.stdout:Modify: 2024-05-08 17:12:43.855681089 +0000 2024-05-08T17:15:42.890 INFO:teuthology.orchestra.run.smithi183.stdout:Change: 2024-05-08 17:12:43.855681089 +0000 2024-05-08T17:15:42.890 INFO:teuthology.orchestra.run.smithi183.stdout: Birth: 2024-05-08 17:12:43.855681089 +0000 2024-05-08T17:15:42.890 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T17:15:42.952 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records in 2024-05-08T17:15:42.952 INFO:teuthology.orchestra.run.smithi183.stderr:1+0 records out 2024-05-08T17:15:42.952 INFO:teuthology.orchestra.run.smithi183.stderr:512 bytes copied, 0.000181904 s, 2.8 MB/s 2024-05-08T17:15:42.953 DEBUG:teuthology.orchestra.run.smithi183:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T17:15:43.009 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:43.010 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T17:15:43.010 INFO:tasks.ceph:Generating config... 2024-05-08T17:15:43.011 INFO:tasks.ceph:[global] lockdep = True 2024-05-08T17:15:43.011 INFO:tasks.ceph:[global] mon client directed command retry = 5 2024-05-08T17:15:43.011 INFO:tasks.ceph:[global] mon election default strategy = 3 2024-05-08T17:15:43.011 INFO:tasks.ceph:[global] ms bind msgr1 = False 2024-05-08T17:15:43.012 INFO:tasks.ceph:[global] ms bind msgr2 = True 2024-05-08T17:15:43.012 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2024-05-08T17:15:43.012 INFO:tasks.ceph:[global] ms type = async 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mgr] debug ms = 1 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] debug mon = 20 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] debug ms = 1 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] debug paxos = 20 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] mon scrub interval = 300 2024-05-08T17:15:43.012 INFO:tasks.ceph:[mon] paxos service trim min = 10 2024-05-08T17:15:43.012 INFO:tasks.ceph:[osd] bluefs allocator = hybrid 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] bluestore allocator = hybrid 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] bluestore zero block detection = True 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug bluefs = 20 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug bluestore = 20 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug monc = 20 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug ms = 1 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug osd = 20 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] debug rocksdb = 10 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-05-08T17:15:43.013 INFO:tasks.ceph:[osd] osd blocked scrub grace period = 3600 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd debug verify cached snaps = True 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd debug verify missing on start = True 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd max markdown count = 1000 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd mclock override recovery settings = True 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd mclock profile = high_recovery_ops 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd op queue = debug_random 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd op queue cut off = debug_random 2024-05-08T17:15:43.014 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2024-05-08T17:15:43.014 INFO:tasks.ceph:Setting up mon.a... 2024-05-08T17:15:43.014 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-05-08T17:15:43.062 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.keyring 2024-05-08T17:15:43.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-05-08T17:15:43.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T17:15:43.174 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.154'), ('mon.c', '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]'), ('mon.b', '172.21.15.183')] 2024-05-08T17:15:43.174 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],172.21.15.183', 'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}, 'mon.c': {}, 'mon.b': {}} 2024-05-08T17:15:43.175 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:43.175 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T17:15:43.230 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.154 --addv c '[v2:172.21.15.154:3301,v1:172.21.15.154:6790]' --add b 172.21.15.183 --print /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:43.316 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:43.316 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: generated fsid c69ba297-505d-4f4e-b390-bd22ef533cfc 2024-05-08T17:15:43.316 INFO:teuthology.orchestra.run.smithi154.stdout:setting min_mon_release = quincy 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:epoch 0 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:fsid c69ba297-505d-4f4e-b390-bd22ef533cfc 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:last_changed 2024-05-08T17:15:43.320397+0000 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:created 2024-05-08T17:15:43.320397+0000 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:min_mon_release 17 (quincy) 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:election_strategy: 3 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:0: [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] mon.a 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:1: [v2:172.21.15.183:3300/0,v1:172.21.15.183:6789/0] mon.b 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:2: [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] mon.c 2024-05-08T17:15:43.317 INFO:teuthology.orchestra.run.smithi154.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2024-05-08T17:15:43.318 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-05-08T17:15:43.336 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c69ba297-505d-4f4e-b390-bd22ef533cfc... 2024-05-08T17:15:43.337 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T17:15:43.379 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-05-08T17:15:43.419 INFO:teuthology.orchestra.run.smithi154.stdout:[global] 2024-05-08T17:15:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: chdir = "" 2024-05-08T17:15:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T17:15:43.419 INFO:teuthology.orchestra.run.smithi154.stdout: auth supported = cephx 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: filestore xattr use omap = true 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: mon clock drift allowed = 1.000 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: osd crush chooseleaf type = 0 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: auth debug = true 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on old message = true 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: ms die on bug = true 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: mon pg warn max object skew = 0 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.420 INFO:teuthology.orchestra.run.smithi154.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default size = 2 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow primary affinity = true 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd allow pg remap = true 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on legacy crush tunables = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on crush straw calc version zero = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on no sortbitwise = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on osd down out interval zero = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon warn on too few osds = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: mon_allow_pool_size_one = true 2024-05-08T17:15:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: osd default data pool replay window = 5 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon allow pool delete = true 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon cluster log file level = debug 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: debug asserts on shutdown = true 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon health detail to clog = false 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon host = "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],172.21.15.183" 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: lockdep = True 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon client directed command retry = 5 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: mon election default strategy = 3 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ms bind msgr1 = False 2024-05-08T17:15:43.422 INFO:teuthology.orchestra.run.smithi154.stdout: ms bind msgr2 = True 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: ms inject socket failures = 5000 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: ms type = async 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: fsid = c69ba297-505d-4f4e-b390-bd22ef533cfc 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout:[osd] 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd journal size = 100 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub load threshold = 5.0 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd scrub max interval = 600 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock profile = high_recovery_ops 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd recover clone overlap = true 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd recovery max chunk = 1048576 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug shutdown = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug op order = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify stray on activate = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug trim objects = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd open classes on start = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug pg log writeout = true 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd deep scrub update digest min age = 30 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: osd map max advance = 10 2024-05-08T17:15:43.424 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: journal zero on create = true 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: filestore ondisk finisher threads = 3 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: filestore apply finisher threads = 3 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bdev debug aio = true 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug misdirected ops = true 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bluefs allocator = hybrid 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore allocator = hybrid 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore block size = 96636764160 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore fsck on mount = True 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: bluestore zero block detection = True 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluefs = 20 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: debug bluestore = 20 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: debug monc = 20 2024-05-08T17:15:43.425 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: debug osd = 20 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: debug rocksdb = 10 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd full ratio = 0.9 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd nearfull ratio = 0.8 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd blocked scrub grace period = 3600 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify cached snaps = True 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd debug verify missing on start = True 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd failsafe full ratio = 0.95 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd max markdown count = 1000 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd mclock override recovery settings = True 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd objectstore = bluestore 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd op queue = debug_random 2024-05-08T17:15:43.426 INFO:teuthology.orchestra.run.smithi154.stdout: osd op queue cut off = debug_random 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: osd shutdown pgref assert = True 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr] 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug mgr = 20 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: mgr/telemetry/nag = false 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout:[mon] 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug ms = 1 2024-05-08T17:15:43.427 INFO:teuthology.orchestra.run.smithi154.stdout: debug mon = 20 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: debug paxos = 20 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: debug auth = 20 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon data avail warn = 5 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon mgr mkfs grace = 240 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min pgs per osd = 4 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd reporter subtree level = osd 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon osd prime pg temp = true 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: mon reweight min bytes per osd = 10 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.428 INFO:teuthology.orchestra.run.smithi154.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: # 1m isn't quite enough 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon_down_mkfs_grace = 2m 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon_warn_on_filestore_osds = false 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon min osdmap epochs = 50 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon osdmap full prune interval = 2 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon osdmap full prune min = 15 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon osdmap full prune txsize = 2 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: mon scrub interval = 300 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: paxos service trim min = 10 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout:[client] 2024-05-08T17:15:43.429 INFO:teuthology.orchestra.run.smithi154.stdout: rgw cache enabled = true 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable ops log = true 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: rgw enable usage log = true 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.a] 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.c] 2024-05-08T17:15:43.430 INFO:teuthology.orchestra.run.smithi154.stdout:[mon.b] 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout:[global] 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: chdir = "" 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: auth supported = cephx 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: filestore xattr use omap = true 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: mon clock drift allowed = 1.000 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: osd crush chooseleaf type = 0 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: auth debug = true 2024-05-08T17:15:43.431 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on old message = true 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: ms die on bug = true 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: mon max pg per osd = 10000 # >= luminous 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: mon pg warn max object skew = 0 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: # disable pg_autoscaler by default for new pools 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: osd_pool_default_pg_autoscale_mode = off 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default size = 2 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.432 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow primary affinity = true 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd allow pg remap = true 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on legacy crush tunables = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on crush straw calc version zero = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on no sortbitwise = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on osd down out interval zero = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon warn on too few osds = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_pool_no_redundancy = false 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: mon_allow_pool_size_one = true 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: osd default data pool replay window = 5 2024-05-08T17:15:43.433 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon allow pool delete = true 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon cluster log file level = debug 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: debug asserts on shutdown = true 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon health detail to clog = false 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon host = "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],172.21.15.183" 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: lockdep = True 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon client directed command retry = 5 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: mon election default strategy = 3 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: ms bind msgr1 = False 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: ms bind msgr2 = True 2024-05-08T17:15:43.434 INFO:teuthology.orchestra.run.smithi183.stdout: ms inject socket failures = 5000 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: ms type = async 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: fsid = c69ba297-505d-4f4e-b390-bd22ef533cfc 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout:[osd] 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd journal size = 100 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub load threshold = 5.0 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd scrub max interval = 600 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock profile = high_recovery_ops 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd recover clone overlap = true 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd recovery max chunk = 1048576 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug shutdown = true 2024-05-08T17:15:43.435 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug op order = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify stray on activate = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug trim objects = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd open classes on start = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug pg log writeout = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd deep scrub update digest min age = 30 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: osd map max advance = 10 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: journal zero on create = true 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: filestore ondisk finisher threads = 3 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: filestore apply finisher threads = 3 2024-05-08T17:15:43.436 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bdev debug aio = true 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug misdirected ops = true 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bluefs allocator = hybrid 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore allocator = hybrid 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore block size = 96636764160 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore fsck on mount = True 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: bluestore zero block detection = True 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluefs = 20 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug bluestore = 20 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug monc = 20 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug osd = 20 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: debug rocksdb = 10 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd backfillfull_ratio = 0.85 2024-05-08T17:15:43.437 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd full ratio = 0.9 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd nearfull ratio = 0.8 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd blocked scrub grace period = 3600 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify cached snaps = True 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd debug verify missing on start = True 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd failsafe full ratio = 0.95 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd max markdown count = 1000 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd mclock override recovery settings = True 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd objectstore = bluestore 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd op queue = debug_random 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd op queue cut off = debug_random 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: osd shutdown pgref assert = True 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr] 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-08T17:15:43.438 INFO:teuthology.orchestra.run.smithi183.stdout: debug mgr = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mgr/telemetry/nag = false 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout:[mon] 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug ms = 1 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug mon = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug paxos = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: debug auth = 20 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon data avail warn = 5 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon mgr mkfs grace = 240 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min pgs per osd = 4 2024-05-08T17:15:43.439 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd reporter subtree level = osd 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon osd prime pg temp = true 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon reweight min bytes per osd = 10 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: auth mon ticket ttl = 660 # 11m 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: auth service ticket ttl = 240 # 4m 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: # don't complain about insecure global_id in the test suite 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: # 1m isn't quite enough 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon_down_mkfs_grace = 2m 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon_warn_on_filestore_osds = false 2024-05-08T17:15:43.440 INFO:teuthology.orchestra.run.smithi183.stdout: mon min osdmap epochs = 50 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: mon osdmap full prune interval = 2 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: mon osdmap full prune min = 15 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: mon osdmap full prune txsize = 2 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: mon scrub interval = 300 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: paxos service trim min = 10 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout:[client] 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: rgw cache enabled = true 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable ops log = true 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: rgw enable usage log = true 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.a] 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.c] 2024-05-08T17:15:43.441 INFO:teuthology.orchestra.run.smithi183.stdout:[mon.b] 2024-05-08T17:15:43.443 INFO:tasks.ceph:Creating admin key on mon.a... 2024-05-08T17:15:43.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-05-08T17:15:43.537 INFO:tasks.ceph:Copying monmap to all nodes... 2024-05-08T17:15:43.537 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:43.537 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-05-08T17:15:43.553 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:43.553 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-05-08T17:15:43.607 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:15:43.608 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:43.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T17:15:43.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T17:15:43.681 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:15:43.682 DEBUG:teuthology.orchestra.run.smithi154:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:43.737 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:15:43.737 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:43.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd of=/etc/ceph/ceph.keyring 2024-05-08T17:15:43.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-05-08T17:15:43.772 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:15:43.772 DEBUG:teuthology.orchestra.run.smithi183:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:43.829 INFO:tasks.ceph:Setting up mon nodes... 2024-05-08T17:15:43.829 INFO:tasks.ceph:Setting up mgr nodes... 2024-05-08T17:15:43.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T17:15:43.879 INFO:teuthology.orchestra.run.smithi154.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2024-05-08T17:15:43.882 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T17:15:43.936 INFO:teuthology.orchestra.run.smithi183.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-05-08T17:15:43.939 INFO:tasks.ceph:Setting up mds nodes... 2024-05-08T17:15:43.939 INFO:tasks.ceph_client:Setting up client nodes... 2024-05-08T17:15:43.939 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T17:15:43.980 INFO:teuthology.orchestra.run.smithi154.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-05-08T17:15:43.993 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2024-05-08T17:15:44.033 INFO:teuthology.orchestra.run.smithi183.stdout:creating /etc/ceph/ceph.client.1.keyring 2024-05-08T17:15:44.046 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-05-08T17:15:44.046 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi154.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi183.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2024-05-08T17:15:44.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-05-08T17:15:44.072 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:44.072 INFO:tasks.ceph:role: osd.0 2024-05-08T17:15:44.072 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:15:44.072 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T17:15:44.143 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:15:44.144 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:15:44.648 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-05-08T17:15:44.651 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-05-08T17:15:44.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-05-08T17:15:44.783 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-05-08T17:15:44.818 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-05-08T17:15:44.884 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:44.884 INFO:tasks.ceph:role: osd.1 2024-05-08T17:15:44.884 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:15:44.884 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:15:44.958 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:15:45.418 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-05-08T17:15:45.420 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-05-08T17:15:45.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2024-05-08T17:15:45.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-05-08T17:15:45.572 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-05-08T17:15:45.637 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:45.637 INFO:tasks.ceph:role: osd.2 2024-05-08T17:15:45.637 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:15:45.637 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T17:15:45.709 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:15:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:15:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:15:45.709 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:15:45.710 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:15:46.197 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-05-08T17:15:46.199 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-05-08T17:15:46.200 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2024-05-08T17:15:46.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-05-08T17:15:46.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-05-08T17:15:46.413 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:46.413 INFO:tasks.ceph:role: osd.3 2024-05-08T17:15:46.414 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:15:46.414 DEBUG:teuthology.orchestra.run.smithi154:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T17:15:46.485 INFO:teuthology.orchestra.run.smithi154.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:15:46.485 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:15:46.485 INFO:teuthology.orchestra.run.smithi154.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:15:46.486 INFO:teuthology.orchestra.run.smithi154.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:15:47.009 INFO:teuthology.orchestra.run.smithi154.stdout:Discarding blocks...Done. 2024-05-08T17:15:47.011 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi154.front.sepia.ceph.com -o noatime 2024-05-08T17:15:47.011 DEBUG:teuthology.orchestra.run.smithi154:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2024-05-08T17:15:47.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-05-08T17:15:47.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:47.284 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.287+0000 7fa486911640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-05-08T17:15:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.288+0000 7fa486911640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-05-08T17:15:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.288+0000 7fa486911640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T17:15:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.288+0000 7fa486911640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T17:15:47.285 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.288+0000 7fa486911640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-05-08T17:15:47.286 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:47.289+0000 7fa486911640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-05-08T17:15:50.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-05-08T17:15:50.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:50.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.258+0000 7f42e9e2c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-05-08T17:15:50.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.258+0000 7f42e9e2c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-05-08T17:15:50.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.258+0000 7f42e9e2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T17:15:50.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.258+0000 7f42e9e2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T17:15:50.255 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.258+0000 7f42e9e2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-05-08T17:15:50.256 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:50.259+0000 7f42e9e2c640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-05-08T17:15:53.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-05-08T17:15:53.145 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.239+0000 7f38f52ff640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.239+0000 7f38f52ff640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.239+0000 7f38f52ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.239+0000 7f38f52ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.239+0000 7f38f52ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-05-08T17:15:53.236 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:53.240+0000 7f38f52ff640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-05-08T17:15:56.088 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-05-08T17:15:56.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.207+0000 7faf0fa1f640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.208+0000 7faf0fa1f640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.208+0000 7faf0fa1f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.208+0000 7faf0fa1f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.208+0000 7faf0fa1f640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-05-08T17:15:56.206 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:15:56.208+0000 7faf0fa1f640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-05-08T17:15:59.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-05-08T17:15:59.082 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2024-05-08T17:15:59.112 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:59.113 INFO:tasks.ceph:role: osd.4 2024-05-08T17:15:59.113 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:15:59.113 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-05-08T17:15:59.189 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:15:59.189 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:15:59.189 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:15:59.190 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:15:59.699 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-08T17:15:59.701 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-08T17:15:59.701 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2024-05-08T17:15:59.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2024-05-08T17:15:59.877 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2024-05-08T17:15:59.944 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T17:15:59.974 INFO:tasks.ceph:role: osd.5 2024-05-08T17:15:59.974 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:15:59.975 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:16:00.017 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:16:00.459 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-08T17:16:00.461 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-08T17:16:00.462 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2024-05-08T17:16:00.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2024-05-08T17:16:00.585 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2024-05-08T17:16:00.649 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T17:16:00.650 INFO:tasks.ceph:role: osd.6 2024-05-08T17:16:00.650 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:16:00.650 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-05-08T17:16:00.722 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:16:00.723 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:16:01.206 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-08T17:16:01.209 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-08T17:16:01.209 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2024-05-08T17:16:01.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2024-05-08T17:16:01.393 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2024-05-08T17:16:01.458 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2024-05-08T17:16:01.458 INFO:tasks.ceph:role: osd.7 2024-05-08T17:16:01.458 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:16:01.458 DEBUG:teuthology.orchestra.run.smithi183:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2024-05-08T17:16:01.530 INFO:teuthology.orchestra.run.smithi183.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2024-05-08T17:16:01.530 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 attr=2, projid32bit=1 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout: = sunit=0 swidth=0 blks 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-05-08T17:16:01.531 INFO:teuthology.orchestra.run.smithi183.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-05-08T17:16:02.031 INFO:teuthology.orchestra.run.smithi183.stdout:Discarding blocks...Done. 2024-05-08T17:16:02.033 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi183.front.sepia.ceph.com -o noatime 2024-05-08T17:16:02.033 DEBUG:teuthology.orchestra.run.smithi183:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2024-05-08T17:16:02.170 DEBUG:teuthology.orchestra.run.smithi183:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2024-05-08T17:16:02.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:02.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.311+0000 7ff29b168640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2024-05-08T17:16:02.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.311+0000 7ff29b168640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2024-05-08T17:16:02.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.311+0000 7ff29b168640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T17:16:02.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.311+0000 7ff29b168640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T17:16:02.308 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.311+0000 7ff29b168640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2024-05-08T17:16:02.309 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:02.312+0000 7ff29b168640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2024-05-08T17:16:05.163 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2024-05-08T17:16:05.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:05.280 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.282+0000 7ffb8de0c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2024-05-08T17:16:05.280 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.283+0000 7ffb8de0c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2024-05-08T17:16:05.280 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.283+0000 7ffb8de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T17:16:05.280 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.283+0000 7ffb8de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T17:16:05.280 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.283+0000 7ffb8de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2024-05-08T17:16:05.281 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:05.283+0000 7ffb8de0c640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2024-05-08T17:16:08.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2024-05-08T17:16:08.160 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:08.252 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.254+0000 7f5ab1d33640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2024-05-08T17:16:08.252 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.255+0000 7f5ab1d33640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2024-05-08T17:16:08.252 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.255+0000 7f5ab1d33640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T17:16:08.253 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.255+0000 7f5ab1d33640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T17:16:08.253 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.255+0000 7f5ab1d33640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2024-05-08T17:16:08.253 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:08.255+0000 7f5ab1d33640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2024-05-08T17:16:11.106 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2024-05-08T17:16:11.132 DEBUG:teuthology.orchestra.run.smithi183:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:11.218 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.220+0000 7f262ac14640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2024-05-08T17:16:11.218 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.220+0000 7f262ac14640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2024-05-08T17:16:11.218 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.221+0000 7f262ac14640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T17:16:11.218 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.221+0000 7f262ac14640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T17:16:11.219 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.221+0000 7f262ac14640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2024-05-08T17:16:11.219 INFO:teuthology.orchestra.run.smithi183.stderr:2024-05-08T17:16:11.221+0000 7f262ac14640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2024-05-08T17:16:14.070 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2024-05-08T17:16:14.099 INFO:tasks.ceph:Reading keys from all nodes... 2024-05-08T17:16:14.100 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2024-05-08T17:16:14.130 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.130 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-05-08T17:16:14.199 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-05-08T17:16:14.268 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.268 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-05-08T17:16:14.336 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.336 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-05-08T17:16:14.402 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.402 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-05-08T17:16:14.431 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.432 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2024-05-08T17:16:14.501 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.501 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2024-05-08T17:16:14.568 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.569 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2024-05-08T17:16:14.636 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.636 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2024-05-08T17:16:14.702 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:14.703 DEBUG:teuthology.orchestra.run.smithi154:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-05-08T17:16:14.721 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:14.722 DEBUG:teuthology.orchestra.run.smithi183:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2024-05-08T17:16:14.758 INFO:tasks.ceph:Adding keys to all mons... 2024-05-08T17:16:14.759 DEBUG:teuthology.orchestra.run.smithi154:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T17:16:14.764 DEBUG:teuthology.orchestra.run.smithi183:> sudo tee -a /etc/ceph/ceph.keyring 2024-05-08T17:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.y] 2024-05-08T17:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA/sztmM9avNBAA6XD1cPMt5C12XXf/WKqY6Q== 2024-05-08T17:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.0] 2024-05-08T17:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBDsztmW1gwERAA1FeW3BzQBPECRCUU78S6sA== 2024-05-08T17:16:14.801 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.1] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBGsztmip5sDxAA6DP/3U4A5R5XSwd1FR2bOw== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.2] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBJsztmZr1JDhAAXOzyiEiQmHAXCW9/XEq0KQ== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.3] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBMsztmh19tDBAA4yBPwzXCaiAZIZXp7m55Pg== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[mgr.x] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA/sztmZAwMOBAAZZZhKZr7VR8Eln1xubSNiA== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.4] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBSsztmMLWUEhAANjP5aksKH4An04wPiFpWkg== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.5] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBVsztmGTnqEBAAh6ETkTfraPO5UfXYoJlHbw== 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.6] 2024-05-08T17:16:14.802 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBYsztmP6dADxAAUBMDmJLBXmVa7rSANx1MNA== 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout:[osd.7] 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBbsztm5P43DRAA8OvNaB2Sira5bXG0aDdo5g== 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout:[client.0] 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQA/sztmatymOhAA1YzAjqG5ofKIBX2AkRcUhw== 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout:[client.1] 2024-05-08T17:16:14.803 INFO:teuthology.orchestra.run.smithi154.stdout: key = AQBAsztm9Es9AhAAc+w5mWhRWvMy3DVIBweueQ== 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.y] 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA/sztmM9avNBAA6XD1cPMt5C12XXf/WKqY6Q== 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.0] 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBDsztmW1gwERAA1FeW3BzQBPECRCUU78S6sA== 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.1] 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBGsztmip5sDxAA6DP/3U4A5R5XSwd1FR2bOw== 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.2] 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBJsztmZr1JDhAAXOzyiEiQmHAXCW9/XEq0KQ== 2024-05-08T17:16:14.822 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.3] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBMsztmh19tDBAA4yBPwzXCaiAZIZXp7m55Pg== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[mgr.x] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA/sztmZAwMOBAAZZZhKZr7VR8Eln1xubSNiA== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.4] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBSsztmMLWUEhAANjP5aksKH4An04wPiFpWkg== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.5] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBVsztmGTnqEBAAh6ETkTfraPO5UfXYoJlHbw== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.6] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBYsztmP6dADxAAUBMDmJLBXmVa7rSANx1MNA== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[osd.7] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBbsztm5P43DRAA8OvNaB2Sira5bXG0aDdo5g== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[client.0] 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQA/sztmatymOhAA1YzAjqG5ofKIBX2AkRcUhw== 2024-05-08T17:16:14.823 INFO:teuthology.orchestra.run.smithi183.stdout:[client.1] 2024-05-08T17:16:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: key = AQBAsztm9Es9AhAAc+w5mWhRWvMy3DVIBweueQ== 2024-05-08T17:16:14.824 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T17:16:14.847 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T17:16:14.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:14.920 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:14.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:14.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.076 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.078 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T17:16:15.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-05-08T17:16:15.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.184 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.287 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.289 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.342 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-05-08T17:16:15.392 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T17:16:15.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T17:16:15.447 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T17:16:15.450 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-05-08T17:16:15.503 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-05-08T17:16:15.503 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-05-08T17:16:15.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T17:16:15.623 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-05-08T17:16:15.647 DEBUG:teuthology.orchestra.run.smithi154:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2024-05-08T17:16:15.712 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T17:16:15.807 DEBUG:teuthology.orchestra.run.smithi154:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2024-05-08T17:16:15.832 DEBUG:teuthology.orchestra.run.smithi183:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2024-05-08T17:16:15.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-05-08T17:16:15.952 DEBUG:teuthology.orchestra.run.smithi183:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2024-05-08T17:16:15.977 DEBUG:teuthology.orchestra.run.smithi154:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:15.979 DEBUG:teuthology.orchestra.run.smithi183:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-05-08T17:16:16.032 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-05-08T17:16:16.033 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T17:16:16.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T17:16:16.035 INFO:tasks.ceph.mon.a:Started 2024-05-08T17:16:16.035 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T17:16:16.035 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T17:16:16.037 INFO:tasks.ceph.mon.c:Started 2024-05-08T17:16:16.037 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T17:16:16.037 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T17:16:16.074 INFO:tasks.ceph.mon.b:Started 2024-05-08T17:16:16.075 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-05-08T17:16:16.075 INFO:tasks.ceph.mgr.y:Restarting daemon 2024-05-08T17:16:16.075 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2024-05-08T17:16:16.078 INFO:tasks.ceph.mgr.y:Started 2024-05-08T17:16:16.078 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-05-08T17:16:16.078 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-05-08T17:16:16.080 INFO:tasks.ceph.mgr.x:Started 2024-05-08T17:16:16.080 INFO:tasks.ceph:Setting crush tunables to default 2024-05-08T17:16:16.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd crush tunables default 2024-05-08T17:16:22.559 INFO:teuthology.orchestra.run.smithi154.stderr:adjusted tunables profile to default 2024-05-08T17:16:22.573 INFO:tasks.ceph:check_enable_crimson: False 2024-05-08T17:16:22.573 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-05-08T17:16:22.573 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:22.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-05-08T17:16:22.599 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:22.599 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-05-08T17:16:22.654 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:22.656+0000 7f71fca11040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T17:16:22.665 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:22.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-05-08T17:16:22.730 DEBUG:teuthology.orchestra.run.smithi154:> set -ex 2024-05-08T17:16:22.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-05-08T17:16:22.743 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:22.746+0000 7f700c442040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T17:16:22.795 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:22.795 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2024-05-08T17:16:22.820 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:22.820 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2024-05-08T17:16:22.833 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:22.836+0000 7f71fca11040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T17:16:22.885 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:22.886 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2024-05-08T17:16:22.887 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:22.889+0000 7f71fca11040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T17:16:22.907 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:22.909+0000 7f700c442040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T17:16:22.931 DEBUG:teuthology.orchestra.run.smithi183:> set -ex 2024-05-08T17:16:22.931 DEBUG:teuthology.orchestra.run.smithi183:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2024-05-08T17:16:22.963 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:22.965+0000 7f700c442040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T17:16:22.987 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858 0 2024-05-08T17:16:23.139 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:23.141+0000 7f71fca11040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T17:16:23.184 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:23.186+0000 7f71fca11040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T17:16:23.210 INFO:teuthology.orchestra.run.smithi183.stdout:0 2024-05-08T17:16:23.221 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 7e66ef3b-5581-4f8c-9807-e16ea4ec972e 1 2024-05-08T17:16:23.267 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:23.270+0000 7f700c442040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T17:16:23.315 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:23.318+0000 7f700c442040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T17:16:23.409 INFO:teuthology.orchestra.run.smithi183.stdout:1 2024-05-08T17:16:23.420 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 86378b92-f863-41ec-a3a5-f1dc78645250 2 2024-05-08T17:16:23.629 INFO:teuthology.orchestra.run.smithi183.stdout:2 2024-05-08T17:16:23.641 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 2fa93019-16da-44c6-ac77-0ef554e24e10 3 2024-05-08T17:16:23.830 INFO:teuthology.orchestra.run.smithi183.stdout:3 2024-05-08T17:16:23.841 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 08750d7a-1ca6-4d68-a62e-893e8d979921 4 2024-05-08T17:16:23.935 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:23.937+0000 7f71fca11040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T17:16:24.023 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.025+0000 7f71fca11040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T17:16:24.040 INFO:teuthology.orchestra.run.smithi183.stdout:4 2024-05-08T17:16:24.052 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 770b03a7-7f15-4427-81ce-e512b68bbb5c 5 2024-05-08T17:16:24.066 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.068+0000 7f71fca11040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T17:16:24.113 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.115+0000 7f700c442040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T17:16:24.204 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.207+0000 7f700c442040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T17:16:24.231 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.233+0000 7f71fca11040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T17:16:24.239 INFO:teuthology.orchestra.run.smithi183.stdout:5 2024-05-08T17:16:24.249 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.252+0000 7f700c442040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T17:16:24.252 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new ac82b5c6-7db8-4383-b0ac-0a203a4d84cd 6 2024-05-08T17:16:24.278 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.282+0000 7f71fca11040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T17:16:24.331 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.333+0000 7f71fca11040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T17:16:24.425 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.428+0000 7f700c442040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T17:16:24.436 INFO:teuthology.orchestra.run.smithi183.stdout:6 2024-05-08T17:16:24.446 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.449+0000 7f71fca11040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T17:16:24.447 DEBUG:teuthology.orchestra.run.smithi183:> sudo ceph --cluster ceph osd new 813f2f3d-68eb-487b-89a1-8b9a5a103485 7 2024-05-08T17:16:24.477 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.479+0000 7f700c442040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T17:16:24.532 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.534+0000 7f700c442040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T17:16:24.633 INFO:teuthology.orchestra.run.smithi183.stdout:7 2024-05-08T17:16:24.644 INFO:tasks.ceph.osd.0:Restarting daemon 2024-05-08T17:16:24.645 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-05-08T17:16:24.647 INFO:tasks.ceph.osd.0:Started 2024-05-08T17:16:24.648 INFO:tasks.ceph.osd.1:Restarting daemon 2024-05-08T17:16:24.648 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-05-08T17:16:24.649 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:24.652+0000 7f700c442040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T17:16:24.650 INFO:tasks.ceph.osd.1:Started 2024-05-08T17:16:24.650 INFO:tasks.ceph.osd.2:Restarting daemon 2024-05-08T17:16:24.650 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-05-08T17:16:24.652 INFO:tasks.ceph.osd.2:Started 2024-05-08T17:16:24.652 INFO:tasks.ceph.osd.3:Restarting daemon 2024-05-08T17:16:24.652 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-05-08T17:16:24.655 INFO:tasks.ceph.osd.3:Started 2024-05-08T17:16:24.655 INFO:tasks.ceph.osd.4:Restarting daemon 2024-05-08T17:16:24.655 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2024-05-08T17:16:24.657 INFO:tasks.ceph.osd.4:Started 2024-05-08T17:16:24.657 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T17:16:24.657 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T17:16:24.659 INFO:tasks.ceph.osd.5:Started 2024-05-08T17:16:24.659 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T17:16:24.659 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T17:16:24.661 INFO:tasks.ceph.osd.6:Started 2024-05-08T17:16:24.661 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T17:16:24.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T17:16:24.664 INFO:tasks.ceph.osd.7:Started 2024-05-08T17:16:24.664 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:24.859 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:24.860 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":10,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:24.632481+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:24.868 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:24.872+0000 7f71fca11040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T17:16:24.871 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-05-08T17:16:24.871 INFO:tasks.ceph:Waiting for OSDs to come up 2024-05-08T17:16:25.058 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.060+0000 7f700c442040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T17:16:25.129 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.132+0000 7f71fca11040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T17:16:25.180 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.183+0000 7f71fca11040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T17:16:25.260 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:25.263+0000 7fcb59625640 -1 Falling back to public interface 2024-05-08T17:16:25.260 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:25.263+0000 7f0ca9212640 -1 Falling back to public interface 2024-05-08T17:16:25.260 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:25.263+0000 7fcb580a4640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6800/0: (98) Address already in use 2024-05-08T17:16:25.262 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:25.265+0000 7f39279b6640 -1 Falling back to public interface 2024-05-08T17:16:25.262 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:25.266+0000 7f614f421640 -1 Falling back to public interface 2024-05-08T17:16:25.276 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:25.279+0000 7f78c5f6c640 -1 Falling back to public interface 2024-05-08T17:16:25.279 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:25.281+0000 7f6df9e18640 -1 Falling back to public interface 2024-05-08T17:16:25.283 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:25.286+0000 7f5a3441f640 -1 Falling back to public interface 2024-05-08T17:16:25.292 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:25.295+0000 7ff99020c640 -1 Falling back to public interface 2024-05-08T17:16:25.298 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.301+0000 7f71fca11040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T17:16:25.318 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.321+0000 7f700c442040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T17:16:25.371 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.374+0000 7f700c442040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T17:16:25.488 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.490+0000 7f700c442040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T17:16:25.589 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.592+0000 7f71fca11040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T17:16:25.637 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.640+0000 7f71fca11040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T17:16:25.676 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:25.778 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.780+0000 7f700c442040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T17:16:25.801 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.804+0000 7f71fca11040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T17:16:25.827 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.829+0000 7f700c442040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T17:16:25.838 INFO:teuthology.misc.health.smithi154.stdout: 2024-05-08T17:16:25.839 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":10,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:24.632481+0000","last_up_change":"0.000000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:25.849 DEBUG:teuthology.misc:0 of 8 OSDs are up 2024-05-08T17:16:25.939 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.943+0000 7f71fca11040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T17:16:25.983 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:25.986+0000 7f700c442040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T17:16:25.988 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:25.991+0000 7f71fca11040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T17:16:26.057 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:26.060+0000 7f71fca11040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T17:16:26.103 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:26.106+0000 7f71fca11040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T17:16:26.121 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.123+0000 7f700c442040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T17:16:26.170 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.173+0000 7f700c442040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T17:16:26.239 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.242+0000 7f700c442040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T17:16:26.284 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.287+0000 7f700c442040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T17:16:26.575 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:26.578+0000 7f71fca11040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T17:16:26.693 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:26.696+0000 7f71fca11040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T17:16:26.764 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.767+0000 7f700c442040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T17:16:26.878 INFO:tasks.ceph.mgr.x.smithi183.stderr:2024-05-08T17:16:26.881+0000 7f700c442040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T17:16:27.707 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:27.711+0000 7f719de50640 -1 mgr.server handle_report got status from non-daemon mon.c 2024-05-08T17:16:27.707 INFO:tasks.ceph.mgr.y.smithi154.stderr:2024-05-08T17:16:27.711+0000 7f719de50640 -1 mgr.server handle_report got status from non-daemon mon.b 2024-05-08T17:16:28.716 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:28.719+0000 7fcb59625640 -1 osd.5 0 log_to_monitors true 2024-05-08T17:16:28.765 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:28.767+0000 7f0ca9212640 -1 osd.6 0 log_to_monitors true 2024-05-08T17:16:28.769 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:28.772+0000 7f6df9e18640 -1 osd.7 0 log_to_monitors true 2024-05-08T17:16:28.787 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:28.791+0000 7f614f421640 -1 osd.3 0 log_to_monitors true 2024-05-08T17:16:28.799 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:28.802+0000 7f5a3441f640 -1 osd.4 0 log_to_monitors true 2024-05-08T17:16:28.806 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:28.810+0000 7ff99020c640 -1 osd.1 0 log_to_monitors true 2024-05-08T17:16:28.807 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:28.810+0000 7f39279b6640 -1 osd.2 0 log_to_monitors true 2024-05-08T17:16:28.807 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:28.810+0000 7f78c5f6c640 -1 osd.0 0 log_to_monitors true 2024-05-08T17:16:30.720 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:30.723+0000 7f0ca4c22640 -1 osd.6 0 waiting for initial osdmap 2024-05-08T17:16:30.721 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:30.724+0000 7f6df5859640 -1 osd.7 0 waiting for initial osdmap 2024-05-08T17:16:30.721 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:30.724+0000 7f5a2fe12640 -1 osd.4 0 waiting for initial osdmap 2024-05-08T17:16:30.723 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:30.726+0000 7f392394c640 -1 osd.2 0 waiting for initial osdmap 2024-05-08T17:16:30.732 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:30.735+0000 7f0ca0a4a640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:30.735 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:30.738+0000 7f5a2bc3a640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:30.747 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:30.750+0000 7f6df1681640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:30.856 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:30.860+0000 7f391f774640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:31.474 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:31.477+0000 7fcb5508b640 -1 osd.5 0 waiting for initial osdmap 2024-05-08T17:16:31.478 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:31.481+0000 7fcb4ceab640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:32.378 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:32.382+0000 7ff98bb2a640 -1 osd.1 0 waiting for initial osdmap 2024-05-08T17:16:32.384 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:32.387+0000 7ff98394a640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:32.397 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:32.400+0000 7f614ae88640 -1 osd.3 0 waiting for initial osdmap 2024-05-08T17:16:32.400 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:32.404+0000 7f6142ca8640 -1 osd.3 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:32.401 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:32.405+0000 7f78c1f02640 -1 osd.0 0 waiting for initial osdmap 2024-05-08T17:16:32.406 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:32.410+0000 7f78b9d22640 -1 osd.0 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:16:32.657 DEBUG:teuthology.orchestra.run.smithi154:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:32.949 INFO:teuthology.misc.health.smithi154.stdout: 2024-05-08T17:16:32.949 INFO:teuthology.misc.health.smithi154.stdout:{"epoch":14,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:32.786322+0000","last_up_change":"2024-05-08T17:16:32.786322+0000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":3448847701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":3448847701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":3448847701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":3448847701}]},"public_addr":"172.21.15.183:6801/3448847701","cluster_addr":"172.21.15.183:6803/3448847701","heartbeat_back_addr":"172.21.15.183:6807/3448847701","heartbeat_front_addr":"172.21.15.183:6805/3448847701","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3679914992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3679914992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3679914992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3679914992}]},"public_addr":"172.21.15.183:6800/3679914992","cluster_addr":"172.21.15.183:6802/3679914992","heartbeat_back_addr":"172.21.15.183:6806/3679914992","heartbeat_front_addr":"172.21.15.183:6804/3679914992","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2695818974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2695818974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2695818974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2695818974}]},"public_addr":"172.21.15.183:6808/2695818974","cluster_addr":"172.21.15.183:6809/2695818974","heartbeat_back_addr":"172.21.15.183:6811/2695818974","heartbeat_front_addr":"172.21.15.183:6810/2695818974","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:32.961 DEBUG:teuthology.misc:8 of 8 OSDs are up 2024-05-08T17:16:32.961 INFO:tasks.ceph:Creating RBD pool 2024-05-08T17:16:32.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd pool create rbd 8 2024-05-08T17:16:33.796 INFO:teuthology.orchestra.run.smithi154.stderr:pool 'rbd' created 2024-05-08T17:16:33.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-05-08T17:16:34.838 INFO:teuthology.orchestra.run.smithi154.stderr:enabled application 'rbd' on pool 'rbd' 2024-05-08T17:16:34.854 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-05-08T17:16:34.855 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T17:16:34.855 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-05-08T17:16:34.855 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-05-08T17:16:34.856 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-05-08T17:16:35.198 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:35.208 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":4,"flags":0,"active_gid":4110,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6816","nonce":867570146}]},"active_addr":"172.21.15.154:6816/867570146","active_change":"2024-05-08T17:16:26.697904+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":4105,"name":"x","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":397872759}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":1026403679}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.154:0","nonce":917760989}]}]} 2024-05-08T17:16:35.220 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-05-08T17:16:35.220 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T17:16:35.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:35.512 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:35.513 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:34.803685+0000","last_up_change":"2024-05-08T17:16:32.786322+0000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":3448847701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":3448847701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":3448847701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":3448847701}]},"public_addr":"172.21.15.183:6801/3448847701","cluster_addr":"172.21.15.183:6803/3448847701","heartbeat_back_addr":"172.21.15.183:6807/3448847701","heartbeat_front_addr":"172.21.15.183:6805/3448847701","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3679914992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3679914992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3679914992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3679914992}]},"public_addr":"172.21.15.183:6800/3679914992","cluster_addr":"172.21.15.183:6802/3679914992","heartbeat_back_addr":"172.21.15.183:6806/3679914992","heartbeat_front_addr":"172.21.15.183:6804/3679914992","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2695818974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2695818974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2695818974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2695818974}]},"public_addr":"172.21.15.183:6808/2695818974","cluster_addr":"172.21.15.183:6809/2695818974","heartbeat_back_addr":"172.21.15.183:6811/2695818974","heartbeat_front_addr":"172.21.15.183:6810/2695818974","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:35.526 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T17:16:35.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:35.815 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:35.815 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":16,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:34.803685+0000","last_up_change":"2024-05-08T17:16:32.786322+0000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":3448847701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":3448847701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":3448847701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":3448847701}]},"public_addr":"172.21.15.183:6801/3448847701","cluster_addr":"172.21.15.183:6803/3448847701","heartbeat_back_addr":"172.21.15.183:6807/3448847701","heartbeat_front_addr":"172.21.15.183:6805/3448847701","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3679914992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3679914992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3679914992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3679914992}]},"public_addr":"172.21.15.183:6800/3679914992","cluster_addr":"172.21.15.183:6802/3679914992","heartbeat_back_addr":"172.21.15.183:6806/3679914992","heartbeat_front_addr":"172.21.15.183:6804/3679914992","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2695818974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2695818974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2695818974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2695818974}]},"public_addr":"172.21.15.183:6808/2695818974","cluster_addr":"172.21.15.183:6809/2695818974","heartbeat_back_addr":"172.21.15.183:6811/2695818974","heartbeat_front_addr":"172.21.15.183:6810/2695818974","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:35.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T17:16:35.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T17:16:35.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T17:16:35.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T17:16:35.831 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T17:16:35.831 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T17:16:35.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T17:16:35.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T17:16:36.084 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:36.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T17:16:36.090 INFO:teuthology.orchestra.run.smithi154.stdout:60129542147 2024-05-08T17:16:36.090 INFO:teuthology.orchestra.run.smithi154.stdout:60129542149 2024-05-08T17:16:36.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T17:16:36.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T17:16:36.092 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:36.092 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T17:16:36.093 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:36.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T17:16:36.094 INFO:teuthology.orchestra.run.smithi154.stdout:60129542147 2024-05-08T17:16:36.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T17:16:36.095 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:36.095 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T17:16:36.096 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:36.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T17:16:36.639 INFO:teuthology.orchestra.run.smithi154.stdout:55834574849 2024-05-08T17:16:36.660 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.7 2024-05-08T17:16:36.702 INFO:teuthology.orchestra.run.smithi154.stdout:55834574849 2024-05-08T17:16:36.721 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:36.724 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574849 for osd.2 2024-05-08T17:16:36.733 INFO:teuthology.orchestra.run.smithi154.stdout:60129542146 2024-05-08T17:16:36.736 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.6 2024-05-08T17:16:36.736 INFO:teuthology.orchestra.run.smithi154.stdout:60129542146 2024-05-08T17:16:36.748 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.0 2024-05-08T17:16:36.751 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:36.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542146 for osd.1 2024-05-08T17:16:36.758 INFO:teuthology.orchestra.run.smithi154.stdout:60129542148 2024-05-08T17:16:36.766 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.5 2024-05-08T17:16:36.775 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:36.776 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542148 for osd.3 2024-05-08T17:16:36.788 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574850 for osd.4 2024-05-08T17:16:37.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T17:16:37.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T17:16:37.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T17:16:37.748 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T17:16:37.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T17:16:37.766 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T17:16:37.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T17:16:37.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T17:16:38.188 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:38.209 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.7 2024-05-08T17:16:38.209 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.273 INFO:teuthology.orchestra.run.smithi154.stdout:60129542147 2024-05-08T17:16:38.287 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.1 2024-05-08T17:16:38.287 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.310 INFO:teuthology.orchestra.run.smithi154.stdout:55834574850 2024-05-08T17:16:38.323 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574850 got 55834574850 for osd.2 2024-05-08T17:16:38.324 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.334 INFO:teuthology.orchestra.run.smithi154.stdout:60129542147 2024-05-08T17:16:38.347 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542147 got 60129542147 for osd.0 2024-05-08T17:16:38.348 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.382 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:38.386 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:38.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.6 2024-05-08T17:16:38.398 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.406 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.5 2024-05-08T17:16:38.406 INFO:teuthology.orchestra.run.smithi154.stdout:60129542149 2024-05-08T17:16:38.406 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.413 INFO:teuthology.orchestra.run.smithi154.stdout:55834574851 2024-05-08T17:16:38.419 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542149 got 60129542149 for osd.3 2024-05-08T17:16:38.419 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.426 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574851 got 55834574851 for osd.4 2024-05-08T17:16:38.426 DEBUG:teuthology.parallel:result is None 2024-05-08T17:16:38.427 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T17:16:38.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:16:38.751 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:38.752 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:16:38.764 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T17:16:38.704049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218788,"kb_used_data":2168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755932,"statfs":{"total":773094113280,"available":772870074368,"internally_reserved":0,"allocated":2220032,"data_stored":1410452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396252+0000","last_change":"2024-05-08T17:16:34.852076+0000","last_active":"2024-05-08T17:16:36.396252+0000","last_peered":"2024-05-08T17:16:36.396252+0000","last_clean":"2024-05-08T17:16:36.396252+0000","last_became_active":"2024-05-08T17:16:34.851798+0000","last_became_peered":"2024-05-08T17:16:34.851798+0000","last_unstale":"2024-05-08T17:16:36.396252+0000","last_undegraded":"2024-05-08T17:16:36.396252+0000","last_fullsized":"2024-05-08T17:16:36.396252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:38.786655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864549+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:16:34.864549+0000","last_peered":"2024-05-08T17:16:34.864549+0000","last_clean":"2024-05-08T17:16:34.864549+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:16:34.864549+0000","last_undegraded":"2024-05-08T17:16:34.864549+0000","last_fullsized":"2024-05-08T17:16:34.864549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:31.840796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396308+0000","last_change":"2024-05-08T17:16:34.861954+0000","last_active":"2024-05-08T17:16:36.396308+0000","last_peered":"2024-05-08T17:16:36.396308+0000","last_clean":"2024-05-08T17:16:36.396308+0000","last_became_active":"2024-05-08T17:16:34.861697+0000","last_became_peered":"2024-05-08T17:16:34.861697+0000","last_unstale":"2024-05-08T17:16:36.396308+0000","last_undegraded":"2024-05-08T17:16:36.396308+0000","last_fullsized":"2024-05-08T17:16:36.396308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:33.117175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864583+0000","last_change":"2024-05-08T17:16:34.862290+0000","last_active":"2024-05-08T17:16:34.864583+0000","last_peered":"2024-05-08T17:16:34.864583+0000","last_clean":"2024-05-08T17:16:34.864583+0000","last_became_active":"2024-05-08T17:16:34.862031+0000","last_became_peered":"2024-05-08T17:16:34.862031+0000","last_unstale":"2024-05-08T17:16:34.864583+0000","last_undegraded":"2024-05-08T17:16:34.864583+0000","last_fullsized":"2024-05-08T17:16:34.864583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:04.366708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.861556+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.861302+0000","last_became_peered":"2024-05-08T17:16:34.861302+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:44.959421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.878806+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:16:35.878806+0000","last_peered":"2024-05-08T17:16:35.878806+0000","last_clean":"2024-05-08T17:16:35.878806+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:16:35.878806+0000","last_undegraded":"2024-05-08T17:16:35.878806+0000","last_fullsized":"2024-05-08T17:16:35.878806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396371+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:16:36.396371+0000","last_peered":"2024-05-08T17:16:36.396371+0000","last_clean":"2024-05-08T17:16:36.396371+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:16:36.396371+0000","last_undegraded":"2024-05-08T17:16:36.396371+0000","last_fullsized":"2024-05-08T17:16:36.396371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.856513+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.856300+0000","last_became_peered":"2024-05-08T17:16:34.856300+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:09.587426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396337+0000","last_change":"2024-05-08T17:16:33.805487+0000","last_active":"2024-05-08T17:16:36.396337+0000","last_peered":"2024-05-08T17:16:36.396337+0000","last_clean":"2024-05-08T17:16:36.396337+0000","last_became_active":"2024-05-08T17:16:33.804407+0000","last_became_peered":"2024-05-08T17:16:33.804407+0000","last_unstale":"2024-05-08T17:16:36.396337+0000","last_undegraded":"2024-05-08T17:16:36.396337+0000","last_fullsized":"2024-05-08T17:16:36.396337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:52.956899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":671744,"data_stored":570110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":574738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:16:38.765 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:16:39.053 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:39.053 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:16:39.066 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T17:16:38.704049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218788,"kb_used_data":2168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755932,"statfs":{"total":773094113280,"available":772870074368,"internally_reserved":0,"allocated":2220032,"data_stored":1410452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396252+0000","last_change":"2024-05-08T17:16:34.852076+0000","last_active":"2024-05-08T17:16:36.396252+0000","last_peered":"2024-05-08T17:16:36.396252+0000","last_clean":"2024-05-08T17:16:36.396252+0000","last_became_active":"2024-05-08T17:16:34.851798+0000","last_became_peered":"2024-05-08T17:16:34.851798+0000","last_unstale":"2024-05-08T17:16:36.396252+0000","last_undegraded":"2024-05-08T17:16:36.396252+0000","last_fullsized":"2024-05-08T17:16:36.396252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:38.786655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864549+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:16:34.864549+0000","last_peered":"2024-05-08T17:16:34.864549+0000","last_clean":"2024-05-08T17:16:34.864549+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:16:34.864549+0000","last_undegraded":"2024-05-08T17:16:34.864549+0000","last_fullsized":"2024-05-08T17:16:34.864549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:31.840796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396308+0000","last_change":"2024-05-08T17:16:34.861954+0000","last_active":"2024-05-08T17:16:36.396308+0000","last_peered":"2024-05-08T17:16:36.396308+0000","last_clean":"2024-05-08T17:16:36.396308+0000","last_became_active":"2024-05-08T17:16:34.861697+0000","last_became_peered":"2024-05-08T17:16:34.861697+0000","last_unstale":"2024-05-08T17:16:36.396308+0000","last_undegraded":"2024-05-08T17:16:36.396308+0000","last_fullsized":"2024-05-08T17:16:36.396308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:33.117175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864583+0000","last_change":"2024-05-08T17:16:34.862290+0000","last_active":"2024-05-08T17:16:34.864583+0000","last_peered":"2024-05-08T17:16:34.864583+0000","last_clean":"2024-05-08T17:16:34.864583+0000","last_became_active":"2024-05-08T17:16:34.862031+0000","last_became_peered":"2024-05-08T17:16:34.862031+0000","last_unstale":"2024-05-08T17:16:34.864583+0000","last_undegraded":"2024-05-08T17:16:34.864583+0000","last_fullsized":"2024-05-08T17:16:34.864583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:04.366708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.861556+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.861302+0000","last_became_peered":"2024-05-08T17:16:34.861302+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:44.959421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.878806+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:16:35.878806+0000","last_peered":"2024-05-08T17:16:35.878806+0000","last_clean":"2024-05-08T17:16:35.878806+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:16:35.878806+0000","last_undegraded":"2024-05-08T17:16:35.878806+0000","last_fullsized":"2024-05-08T17:16:35.878806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396371+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:16:36.396371+0000","last_peered":"2024-05-08T17:16:36.396371+0000","last_clean":"2024-05-08T17:16:36.396371+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:16:36.396371+0000","last_undegraded":"2024-05-08T17:16:36.396371+0000","last_fullsized":"2024-05-08T17:16:36.396371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.856513+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.856300+0000","last_became_peered":"2024-05-08T17:16:34.856300+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:09.587426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396337+0000","last_change":"2024-05-08T17:16:33.805487+0000","last_active":"2024-05-08T17:16:36.396337+0000","last_peered":"2024-05-08T17:16:36.396337+0000","last_clean":"2024-05-08T17:16:36.396337+0000","last_became_active":"2024-05-08T17:16:33.804407+0000","last_became_peered":"2024-05-08T17:16:33.804407+0000","last_unstale":"2024-05-08T17:16:36.396337+0000","last_undegraded":"2024-05-08T17:16:36.396337+0000","last_fullsized":"2024-05-08T17:16:36.396337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:52.956899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":671744,"data_stored":570110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":574738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:16:39.067 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T17:16:39.067 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T17:16:39.067 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-05-08T17:16:39.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-05-08T17:16:39.389 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:39.389 INFO:teuthology.orchestra.run.smithi154.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T17:16:39.402 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-05-08T17:16:39.403 INFO:teuthology.run_tasks:Running task thrashosds... 2024-05-08T17:16:39.414 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0} 2024-05-08T17:16:39.415 INFO:tasks.thrashosds:overrides is {'admin_socket': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid'}, 'ceph': {'conf': {'global': {'lockdep': True, 'mon client directed command retry': 5, 'mon election default strategy': 3, 'ms bind msgr1': False, 'ms bind msgr2': True, 'ms inject socket failures': 5000, 'ms type': 'async'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'mon scrub interval': 300, 'paxos service trim min': 10}, 'osd': {'bluefs allocator': 'hybrid', 'bluestore allocator': 'hybrid', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'bluestore zero block detection': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug monc': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd blocked scrub grace period': 3600, 'osd debug verify cached snaps': True, 'osd debug verify missing on start': True, 'osd failsafe full ratio': 0.95, 'osd max markdown count': 1000, 'osd mclock override recovery settings': True, 'osd mclock profile': 'high_recovery_ops', 'osd objectstore': 'bluestore', 'osd op queue': 'debug_random', 'osd op queue cut off': 'debug_random', 'osd shutdown pgref assert': True}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'but it is still running', 'objects unfound and apparently lost', 'overall HEALTH_', '\\(OSDMAP_FLAGS\\)', '\\(OSD_', '\\(PG_', '\\(POOL_', '\\(CACHE_POOL_', '\\(SMALLER_PGP_NUM\\)', '\\(OBJECT_', '\\(SLOW_OPS\\)', '\\(REQUEST_SLOW\\)', '\\(TOO_FEW_PGS\\)', 'slow request', 'timeout on replica', 'late reservation from', 'MON_DOWN', 'OSDMAP_FLAGS', 'OSD_DOWN', 'PG_DEGRADED', 'PG_AVAILABILITY', 'POOL_APP_NOT_ENABLED', '\\(OSD_SLOW_PING_TIME', 'overall HEALTH_', '\\(MON_DOWN\\)', '\\(OSDMAP_FLAGS\\)', '\\(SMALLER_PGP_NUM\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW OPS\\)', 'slow request'], 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-lflores-testing-2-2024-05-07-1606-squid', 'sha1': '95a4ad3c90f2f12f920e87725ad8d9002f948722'}} 2024-05-08T17:16:39.415 INFO:tasks.thrashosds:config is {'chance_pgnum_grow': 1, 'chance_pgnum_shrink': 1, 'chance_pgpnum_fix': 1, 'timeout': 1200, 'sighup_delay': 0.1, 'optrack_toggle_delay': 2.0, 'dump_ops_enable': 'true', 'noscrub_toggle_delay': 2.0, 'random_eio': 0.0, 'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5} 2024-05-08T17:16:39.415 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr_debug_aggressive_pg_num_changes true 2024-05-08T17:16:39.722 INFO:tasks.thrashosds:Beginning thrashosds... 2024-05-08T17:16:39.722 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T17:16:39.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:16:40.008 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:40.008 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:16:40.020 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T17:16:38.704049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218788,"kb_used_data":2168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755932,"statfs":{"total":773094113280,"available":772870074368,"internally_reserved":0,"allocated":2220032,"data_stored":1410452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396252+0000","last_change":"2024-05-08T17:16:34.852076+0000","last_active":"2024-05-08T17:16:36.396252+0000","last_peered":"2024-05-08T17:16:36.396252+0000","last_clean":"2024-05-08T17:16:36.396252+0000","last_became_active":"2024-05-08T17:16:34.851798+0000","last_became_peered":"2024-05-08T17:16:34.851798+0000","last_unstale":"2024-05-08T17:16:36.396252+0000","last_undegraded":"2024-05-08T17:16:36.396252+0000","last_fullsized":"2024-05-08T17:16:36.396252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:38.786655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864549+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:16:34.864549+0000","last_peered":"2024-05-08T17:16:34.864549+0000","last_clean":"2024-05-08T17:16:34.864549+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:16:34.864549+0000","last_undegraded":"2024-05-08T17:16:34.864549+0000","last_fullsized":"2024-05-08T17:16:34.864549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:31.840796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396308+0000","last_change":"2024-05-08T17:16:34.861954+0000","last_active":"2024-05-08T17:16:36.396308+0000","last_peered":"2024-05-08T17:16:36.396308+0000","last_clean":"2024-05-08T17:16:36.396308+0000","last_became_active":"2024-05-08T17:16:34.861697+0000","last_became_peered":"2024-05-08T17:16:34.861697+0000","last_unstale":"2024-05-08T17:16:36.396308+0000","last_undegraded":"2024-05-08T17:16:36.396308+0000","last_fullsized":"2024-05-08T17:16:36.396308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:33.117175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864583+0000","last_change":"2024-05-08T17:16:34.862290+0000","last_active":"2024-05-08T17:16:34.864583+0000","last_peered":"2024-05-08T17:16:34.864583+0000","last_clean":"2024-05-08T17:16:34.864583+0000","last_became_active":"2024-05-08T17:16:34.862031+0000","last_became_peered":"2024-05-08T17:16:34.862031+0000","last_unstale":"2024-05-08T17:16:34.864583+0000","last_undegraded":"2024-05-08T17:16:34.864583+0000","last_fullsized":"2024-05-08T17:16:34.864583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:04.366708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.861556+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.861302+0000","last_became_peered":"2024-05-08T17:16:34.861302+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:44.959421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.878806+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:16:35.878806+0000","last_peered":"2024-05-08T17:16:35.878806+0000","last_clean":"2024-05-08T17:16:35.878806+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:16:35.878806+0000","last_undegraded":"2024-05-08T17:16:35.878806+0000","last_fullsized":"2024-05-08T17:16:35.878806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396371+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:16:36.396371+0000","last_peered":"2024-05-08T17:16:36.396371+0000","last_clean":"2024-05-08T17:16:36.396371+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:16:36.396371+0000","last_undegraded":"2024-05-08T17:16:36.396371+0000","last_fullsized":"2024-05-08T17:16:36.396371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.856513+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.856300+0000","last_became_peered":"2024-05-08T17:16:34.856300+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:09.587426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396337+0000","last_change":"2024-05-08T17:16:33.805487+0000","last_active":"2024-05-08T17:16:36.396337+0000","last_peered":"2024-05-08T17:16:36.396337+0000","last_clean":"2024-05-08T17:16:36.396337+0000","last_became_active":"2024-05-08T17:16:33.804407+0000","last_became_peered":"2024-05-08T17:16:33.804407+0000","last_unstale":"2024-05-08T17:16:36.396337+0000","last_undegraded":"2024-05-08T17:16:36.396337+0000","last_fullsized":"2024-05-08T17:16:36.396337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:52.956899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":671744,"data_stored":570110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":574738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:16:40.021 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:16:40.302 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:40.303 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:16:40.314 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2024-05-08T17:16:38.704049+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":218788,"kb_used_data":2168,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755932,"statfs":{"total":773094113280,"available":772870074368,"internally_reserved":0,"allocated":2220032,"data_stored":1410452,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":9,"apply_latency_ms":9,"commit_latency_ns":9000000,"apply_latency_ns":9000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"2.000356"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396252+0000","last_change":"2024-05-08T17:16:34.852076+0000","last_active":"2024-05-08T17:16:36.396252+0000","last_peered":"2024-05-08T17:16:36.396252+0000","last_clean":"2024-05-08T17:16:36.396252+0000","last_became_active":"2024-05-08T17:16:34.851798+0000","last_became_peered":"2024-05-08T17:16:34.851798+0000","last_unstale":"2024-05-08T17:16:36.396252+0000","last_undegraded":"2024-05-08T17:16:36.396252+0000","last_fullsized":"2024-05-08T17:16:36.396252+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:38.786655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864549+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:16:34.864549+0000","last_peered":"2024-05-08T17:16:34.864549+0000","last_clean":"2024-05-08T17:16:34.864549+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:16:34.864549+0000","last_undegraded":"2024-05-08T17:16:34.864549+0000","last_fullsized":"2024-05-08T17:16:34.864549+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:53:31.840796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396308+0000","last_change":"2024-05-08T17:16:34.861954+0000","last_active":"2024-05-08T17:16:36.396308+0000","last_peered":"2024-05-08T17:16:36.396308+0000","last_clean":"2024-05-08T17:16:36.396308+0000","last_became_active":"2024-05-08T17:16:34.861697+0000","last_became_peered":"2024-05-08T17:16:34.861697+0000","last_unstale":"2024-05-08T17:16:36.396308+0000","last_undegraded":"2024-05-08T17:16:36.396308+0000","last_fullsized":"2024-05-08T17:16:36.396308+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:21:33.117175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":16,"state":"active+clean","last_fresh":"2024-05-08T17:16:34.864583+0000","last_change":"2024-05-08T17:16:34.862290+0000","last_active":"2024-05-08T17:16:34.864583+0000","last_peered":"2024-05-08T17:16:34.864583+0000","last_clean":"2024-05-08T17:16:34.864583+0000","last_became_active":"2024-05-08T17:16:34.862031+0000","last_became_peered":"2024-05-08T17:16:34.862031+0000","last_unstale":"2024-05-08T17:16:34.864583+0000","last_undegraded":"2024-05-08T17:16:34.864583+0000","last_fullsized":"2024-05-08T17:16:34.864583+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:54:04.366708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.861556+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.861302+0000","last_became_peered":"2024-05-08T17:16:34.861302+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:22:44.959421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.878806+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:16:35.878806+0000","last_peered":"2024-05-08T17:16:35.878806+0000","last_clean":"2024-05-08T17:16:35.878806+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:16:35.878806+0000","last_undegraded":"2024-05-08T17:16:35.878806+0000","last_fullsized":"2024-05-08T17:16:35.878806+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396371+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:16:36.396371+0000","last_peered":"2024-05-08T17:16:36.396371+0000","last_clean":"2024-05-08T17:16:36.396371+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:16:36.396371+0000","last_undegraded":"2024-05-08T17:16:36.396371+0000","last_fullsized":"2024-05-08T17:16:36.396371+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:35.873279+0000","last_change":"2024-05-08T17:16:34.856513+0000","last_active":"2024-05-08T17:16:35.873279+0000","last_peered":"2024-05-08T17:16:35.873279+0000","last_clean":"2024-05-08T17:16:35.873279+0000","last_became_active":"2024-05-08T17:16:34.856300+0000","last_became_peered":"2024-05-08T17:16:34.856300+0000","last_unstale":"2024-05-08T17:16:35.873279+0000","last_undegraded":"2024-05-08T17:16:35.873279+0000","last_fullsized":"2024-05-08T17:16:35.873279+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:41:09.587426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":164,"reported_epoch":17,"state":"active+clean","last_fresh":"2024-05-08T17:16:36.396337+0000","last_change":"2024-05-08T17:16:33.805487+0000","last_active":"2024-05-08T17:16:36.396337+0000","last_peered":"2024-05-08T17:16:36.396337+0000","last_clean":"2024-05-08T17:16:36.396337+0000","last_became_active":"2024-05-08T17:16:33.804407+0000","last_became_peered":"2024-05-08T17:16:33.804407+0000","last_unstale":"2024-05-08T17:16:36.396337+0000","last_undegraded":"2024-05-08T17:16:36.396337+0000","last_fullsized":"2024-05-08T17:16:36.396337+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:01:52.956899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1064960,"data_stored":1057856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542149,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":136,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":139264,"data_stored":41182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542147,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28120,"kb_used_data":656,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343720,"statfs":{"total":96636764160,"available":96607969280,"internally_reserved":0,"allocated":671744,"data_stored":570110,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574851,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27684,"kb_used_data":664,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344156,"statfs":{"total":96636764160,"available":96608415744,"internally_reserved":0,"allocated":679936,"data_stored":574738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574852,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574852,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574851,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":144,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":147456,"data_stored":45810,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:16:40.315 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T17:16:40.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump 2024-05-08T17:16:40.599 INFO:teuthology.orchestra.run.smithi154.stdout:epoch 17 2024-05-08T17:16:40.599 INFO:teuthology.orchestra.run.smithi154.stdout:fsid c69ba297-505d-4f4e-b390-bd22ef533cfc 2024-05-08T17:16:40.599 INFO:teuthology.orchestra.run.smithi154.stdout:created 2024-05-08T17:16:22.335952+0000 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:modified 2024-05-08T17:16:35.854143+0000 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:crush_version 4 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:full_ratio 0.95 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:backfillfull_ratio 0.9 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:nearfull_ratio 0.85 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:require_min_compat_client luminous 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:min_compat_client jewel 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:require_osd_release squid 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:stretch_mode_enabled false 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:pool 1 '.mgr' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 16 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 8.00 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:pool 2 'rbd' replicated size 2 min_size 1 crush_rule 0 object_hash rjenkins pg_num 8 pgp_num 8 autoscale_mode off last_change 17 flags hashpspool stripe_width 0 application rbd read_balance_score 1.99 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:max_osd 8 2024-05-08T17:16:40.600 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6808/3590430155 v2:172.21.15.154:6809/3590430155 exists,up c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6812/2114037860 v2:172.21.15.154:6813/2114037860 exists,up 7e66ef3b-5581-4f8c-9807-e16ea4ec972e 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6800/2790378686 v2:172.21.15.154:6801/2790378686 exists,up 86378b92-f863-41ec-a3a5-f1dc78645250 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6804/3602122863 v2:172.21.15.154:6805/3602122863 exists,up 2fa93019-16da-44c6-ac77-0ef554e24e10 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6812/1597208930 v2:172.21.15.183:6813/1597208930 exists,up 08750d7a-1ca6-4d68-a62e-893e8d979921 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6801/3448847701 v2:172.21.15.183:6803/3448847701 exists,up 770b03a7-7f15-4427-81ce-e512b68bbb5c 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6800/3679914992 v2:172.21.15.183:6802/3679914992 exists,up ac82b5c6-7db8-4383-b0ac-0a203a4d84cd 2024-05-08T17:16:40.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6808/2695818974 v2:172.21.15.183:6809/2695818974 exists,up 813f2f3d-68eb-487b-89a1-8b9a5a103485 2024-05-08T17:16:40.612 INFO:tasks.ceph.ceph_manager.ceph:['osd.0 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6808/3590430155 v2:172.21.15.154:6809/3590430155 exists,up c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858', 'osd.1 up in weight 1 up_from 14 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6812/2114037860 v2:172.21.15.154:6813/2114037860 exists,up 7e66ef3b-5581-4f8c-9807-e16ea4ec972e', 'osd.2 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6800/2790378686 v2:172.21.15.154:6801/2790378686 exists,up 86378b92-f863-41ec-a3a5-f1dc78645250', 'osd.3 up in weight 1 up_from 14 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.154:6804/3602122863 v2:172.21.15.154:6805/3602122863 exists,up 2fa93019-16da-44c6-ac77-0ef554e24e10', 'osd.4 up in weight 1 up_from 13 up_thru 0 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6812/1597208930 v2:172.21.15.183:6813/1597208930 exists,up 08750d7a-1ca6-4d68-a62e-893e8d979921', 'osd.5 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6801/3448847701 v2:172.21.15.183:6803/3448847701 exists,up 770b03a7-7f15-4427-81ce-e512b68bbb5c', 'osd.6 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6800/3679914992 v2:172.21.15.183:6802/3679914992 exists,up ac82b5c6-7db8-4383-b0ac-0a203a4d84cd', 'osd.7 up in weight 1 up_from 13 up_thru 15 down_at 0 last_clean_interval [0,0) v2:172.21.15.183:6808/2695818974 v2:172.21.15.183:6809/2695818974 exists,up 813f2f3d-68eb-487b-89a1-8b9a5a103485'] 2024-05-08T17:16:40.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mon.a.asok config show 2024-05-08T17:16:40.713 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "mon.a", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster": "ceph", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "admin_socket": "/var/run/ceph/ceph-mon.a.asok", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "admin_socket_mode": "", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_allow_insecure_global_id_reclaim": "true", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_client_required": "cephx, none", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_cluster_required": "cephx", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_debug": "true", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_expose_insecure_global_id_reclaim": "true", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_mon_ticket_ttl": "660.000000", 2024-05-08T17:16:40.714 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_service_required": "cephx", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_service_ticket_ttl": "240.000000", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "auth_supported": "cephx", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_aio": "true", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_aio_max_queue_depth": "1024", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_aio_poll_ms": "250", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_aio_reap_max": "16", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_async_discard": "false", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_block_size": "4096", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_debug_aio": "false", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_debug_aio_log_age": "5.000000", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_debug_aio_suicide_timeout": "60.000000", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_debug_inflight_ios": "false", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_enable_discard": "false", 2024-05-08T17:16:40.715 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_flock_retry": "3", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_flock_retry_interval": "0.100000", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_inject_crash": "0", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_inject_crash_flush_delay": "2", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_ioring": "false", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_ioring_hipri": "false", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_ioring_sqthread_poll": "false", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_nvme_unbind_from_kernel": "false", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_read_buffer_alignment": "4096", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_read_preallocated_huge_buffers": "", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bdev_type": "", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_alloc_size": "1048576", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_allocator": "hybrid", 2024-05-08T17:16:40.716 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_buffered_io": "true", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_check_for_zeros": "false", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_check_volume_selector_often": "false", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_check_volume_selector_on_umount": "false", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_compact_log_sync": "false", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_failed_shared_alloc_cooldown": "600.000000", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_log_compact_min_ratio": "5.000000", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_log_compact_min_size": "16777216", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_log_replay_check_allocations": "true", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_max_log_runway": "4194304", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_max_prefetch": "1048576", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_min_flush_size": "524288", 2024-05-08T17:16:40.717 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_min_log_runway": "1048576", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_replay_recovery": "false", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_replay_recovery_disable_compact": "false", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_shared_alloc_size": "65536", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluefs_sync_write": "false", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_2q_cache_kin_ratio": "0.500000", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_2q_cache_kout_ratio": "0.500000", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_alloc_stats_dump_interval": "86400.000000", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_allocation_from_file": "true", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_allocator": "hybrid", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_avl_alloc_bf_free_pct": "4", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_avl_alloc_bf_threshold": "131072", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_avl_alloc_ff_max_search_bytes": "16777216", 2024-05-08T17:16:40.718 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_avl_alloc_ff_max_search_count": "100", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bitmapallocator_blocks_per_zone": "1024", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bitmapallocator_span_size": "1024", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_blobid_prealloc": "10240", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_create": "true", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_db_create": "false", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_db_path": "", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_db_size": "0", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_path": "", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_preallocate_file": "false", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_size": "107374182400", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_wal_create": "false", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_wal_path": "", 2024-05-08T17:16:40.719 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_block_wal_size": "100663296", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bluefs": "true", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bluefs_alloc_failure_dump_interval": "0.000000", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bluefs_env_mirror": "false", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_bluefs_max_free": "10737418240", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_age_bin_interval": "1.000000", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_age_bins_data": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_age_bins_kv": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_age_bins_kv_onode": "0 0 0 0 0 0 0 0 0 720", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_age_bins_meta": "1 2 6 24 120 720 0 0 0 0", 2024-05-08T17:16:40.720 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_autotune": "true", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_autotune_interval": "5.000000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_kv_onode_ratio": "0.040000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_kv_ratio": "0.450000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_meta_ratio": "0.450000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_size": "0", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_size_hdd": "1073741824", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_size_ssd": "3221225472", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_trim_interval": "0.050000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_trim_max_skip_pinned": "1000", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cache_type": "2q", 2024-05-08T17:16:40.721 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_cleaner_sleep_interval": "5.000000", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_clone_cow": "true", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_algorithm": "snappy", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_max_blob_size": "0", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_max_blob_size_hdd": "65536", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_max_blob_size_ssd": "65536", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_min_blob_size": "0", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_min_blob_size_hdd": "8192", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_min_blob_size_ssd": "65536", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_mode": "none", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_compression_required_ratio": "0.875000", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_csum_type": "crc32c", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_enforce_settings": "default", 2024-05-08T17:16:40.722 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_freelist": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_fsck_abort": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_inject_allocation_from_file_failure": "0.000000", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_inject_csum_err_probability": "0.000000", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_inject_read_err": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_legacy_omap": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_no_reuse_blocks": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_omit_block_device_write": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_omit_kv_commit": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_permit_any_bdev_label": "false", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_prefill": "0.000000", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_prefragment_max": "1048576", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_random_read_err": "0.000000", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_randomize_serial_transaction": "0", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_small_allocations": "0", 2024-05-08T17:16:40.723 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_debug_too_many_blobs_threshold": "24576", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_default_buffered_read": "true", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_default_buffered_write": "false", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_deferred_batch_ops": "0", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_deferred_batch_ops_hdd": "64", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_deferred_batch_ops_ssd": "16", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_elastic_shared_blobs": "true", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_map_inline_shard_prealloc_size": "256", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_map_shard_max_size": "1200", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_map_shard_min_size": "150", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_map_shard_target_size": "500", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_extent_map_shard_target_size_slop": "0.200000", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fail_eio": "false", 2024-05-08T17:16:40.724 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_freelist_blocks_per_key": "128", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_error_on_no_per_pg_omap": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_error_on_no_per_pool_omap": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_error_on_no_per_pool_stats": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_mkfs": "true", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_mkfs_deep": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_mount": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_mount_deep": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_umount": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_on_umount_deep": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_quick_fix_on_mount": "false", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_quick_fix_threads": "2", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_read_bytes_cap": "67108864", 2024-05-08T17:16:40.725 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_fsck_shared_blob_tracker_size": "0.031250", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_gc_enable_blob_threshold": "0", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_gc_enable_total_threshold": "0", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_hybrid_alloc_mem_cap": "67108864", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_ignore_data_csum": "false", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_kv_sync_util_logging_s": "10.000000", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_kvbackend": "rocksdb", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_log_collection_list_age": "60.000000", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_log_omap_iterator_age": "5.000000", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_log_op_age": "5.000000", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_alloc_size": "0", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_blob_size": "0", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_blob_size_hdd": "65536", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_blob_size_ssd": "65536", 2024-05-08T17:16:40.726 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_defer_interval": "3.000000", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_max_deferred_txc": "32", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_min_alloc_size": "0", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_min_alloc_size_hdd": "4096", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_min_alloc_size_ssd": "4096", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_nid_prealloc": "1024", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_prefer_deferred_size": "0", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_prefer_deferred_size_hdd": "65536", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_prefer_deferred_size_ssd": "0", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_qfsck_on_mount": "true", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_retry_disk_reads": "3", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_rocksdb_cf": "true", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_rocksdb_cfs": "m(3) p(3,0-12) O(3,0-13)=block_cache={type=binned_lru} L=min_write_buffer_number_to_merge=32 P=min_write_buffer_number_to_merge=32", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_rocksdb_options": "compression=kLZ4Compression,max_write_buffer_number=64,min_write_buffer_number_to_merge=6,compaction_style=kCompactionStyleLevel,write_buffer_size=16777216,max_background_jobs=4,level0_file_num_compaction_trigger=8,max_bytes_for_level_base=1073741824,max_bytes_for_level_multiplier=8,compaction_readahead_size=2MB,max_total_wal_size=1073741824,writable_file_max_buffer_size=0", 2024-05-08T17:16:40.727 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_rocksdb_options_annex": "", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_spdk_coremask": "0x1", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_spdk_io_sleep": "5", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_spdk_max_io_completion": "0", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_spdk_mem": "512", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_sync_submit_transaction": "false", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_bytes": "67108864", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_cost_per_io": "0", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_cost_per_io_hdd": "670000", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_cost_per_io_ssd": "4000", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_deferred_bytes": "134217728", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_throttle_trace_rate": "0.000000", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_tracing": "false", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_use_optimal_io_size_for_min_alloc_size": "false", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_volume_selection_policy": "use_some_extra", 2024-05-08T17:16:40.728 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_volume_selection_reserved": "0", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_volume_selection_reserved_factor": "2.000000", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_warn_on_bluefs_spillover": "true", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_warn_on_legacy_statfs": "true", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_warn_on_no_per_pg_omap": "false", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_warn_on_no_per_pool_omap": "true", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_warn_on_spurious_read_errors": "true", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "bluestore_zero_block_detection": "false", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephadm_path": "/usr/sbin/cephadm", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_action_update_interval": "2", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_directory_scan_interval": "10", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_max_concurrent_directory_syncs": "3", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_max_consecutive_failures_per_directory": "10", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_max_snapshot_sync_per_cycle": "3", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_mount_timeout": "10", 2024-05-08T17:16:40.729 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_perf_stats_prio": "5", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_restart_mirror_on_blocklist_interval": "30", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_restart_mirror_on_failure_interval": "20", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephfs_mirror_retry_failed_directories_interval": "60", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephsqlite_blocklist_dead_locker": "true", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephsqlite_lock_renewal_interval": "2000", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephsqlite_lock_renewal_timeout": "30000", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_cluster_require_signatures": "false", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_cluster_require_version": "2", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_require_signatures": "false", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_require_version": "2", 2024-05-08T17:16:40.730 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_service_require_signatures": "false", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_service_require_version": "2", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "cephx_sign_messages": "true", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "chdir": "", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_acl_type": "", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_asio_thread_count": "2", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_cache_mid": "0.750000", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_cache_size": "16384", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_caps_release_delay": "5", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_check_pool_perm": "true", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_collect_and_send_global_metrics": "false", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_debug_force_sync_read": "false", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_debug_getattr_caps": "false", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_debug_inject_tick_delay": "0", 2024-05-08T17:16:40.731 INFO:teuthology.orchestra.run.smithi154.stdout: "client_die_on_failed_dentry_invalidate": "true", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_die_on_failed_remount": "false", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_dirsize_rbytes": "true", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_force_lazyio": "false", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_fs": "", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_inject_fixed_oldest_tid": "false", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_inject_release_failure": "false", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_max_inline_size": "4096", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_max_retries_on_remount_failure": "5", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_mds_namespace": "", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_metadata": "", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_mount_gid": "-1", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_mount_timeout": "300", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_mount_uid": "-1", 2024-05-08T17:16:40.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_mountpoint": "/", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_notify_timeout": "10", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc": "true", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc_max_dirty": "104857600", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc_max_dirty_age": "5.000000", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc_max_objects": "1000", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc_size": "209715200", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_oc_target_dirty": "8388608", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_permissions": "true", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_quota": "true", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_quota_df": "true", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_readahead_max_bytes": "0", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_readahead_max_periods": "4", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_readahead_min": "131072", 2024-05-08T17:16:40.733 INFO:teuthology.orchestra.run.smithi154.stdout: "client_reconnect_stale": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_shutdown_timeout": "30", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_snapdir": ".snap", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_tick_interval": "1", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_trace": "", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_try_dentry_invalidate": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_use_faked_inos": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "client_use_random_mds": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_graylog": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_graylog_host": "127.0.0.1", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_graylog_port": "12201", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_monitors": "default=true", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_syslog": "false", 2024-05-08T17:16:40.734 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_syslog_facility": "default=daemon audit=local0", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "clog_to_syslog_level": "info", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "-", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_network": "", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_network_interface": "", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "compressor_zlib_isal": "false", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "compressor_zlib_level": "5", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "compressor_zlib_winsize": "-15", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "compressor_zstd_level": "1", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "container_image": "docker.io/ceph/daemon-base:latest-master-devel", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "crash_dir": "/var/lib/ceph/crash", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "crimson_alien_op_num_threads": "6", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "crimson_alien_thread_cpu_cores": "", 2024-05-08T17:16:40.735 INFO:teuthology.orchestra.run.smithi154.stdout: "crimson_osd_obc_lru_size": "512", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "crimson_osd_scheduler_concurrency": "0", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "crimson_seastar_cpu_cores": "", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location_hook": "", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location_hook_timeout": "10", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "daemonize": "false", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "daos_pool": "tank", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "dbstore_config_uri": "file:/var/lib/ceph/radosgw/dbstore-config.db", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "dbstore_db_dir": "/var/lib/ceph/radosgw", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "dbstore_db_name_prefix": "dbstore", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_alienstore": "0/5", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_allow_any_pool_priority": "false", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_asok": "1/5", 2024-05-08T17:16:40.736 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_asok_assert_abort": "false", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_asserts_on_shutdown": "true", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_auth": "20/20", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_bdev": "1/3", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_bluefs": "1/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_bluestore": "1/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_buffer": "0/1", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_ceph_exporter": "1/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_cephfs_mirror": "0/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_cephsqlite": "0/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_client": "0/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_compressor": "1/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_context": "0/1", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_crimson_interrupt": "0/5", 2024-05-08T17:16:40.737 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_crush": "1/1", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_crypto": "1/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_cyanstore": "0/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_deliberately_leak_memory": "false", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_disable_randomized_ping": "false", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_dpdk": "1/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_eventtrace": "1/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_filer": "0/1", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_filestore": "1/3", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_finisher": "1/1", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_fuse": "1/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_heartbeat_testing_span": "0", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_heartbeatmap": "1/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_immutable_obj_cache": "0/5", 2024-05-08T17:16:40.738 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_javaclient": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_journal": "1/3", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_journaler": "0/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_kstore": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_lockdep": "0/1", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mclock": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_balancer": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_locker": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_log": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_log_expire": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_migrator": "1/5", 2024-05-08T17:16:40.739 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mds_quiesce": "3/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_memstore": "1/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mgr": "2/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mgrc": "1/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_mon": "20/20", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_monc": "0/10", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_ms": "1/1", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_none": "0/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_objclass": "0/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_objectcacher": "0/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_objecter": "0/1", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_optracker": "0/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_osd": "1/5", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_paxos": "20/20", 2024-05-08T17:16:40.740 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_perfcounter": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_prioritycache": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rados": "0/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rbd": "0/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rbd_mirror": "0/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rbd_pwl": "0/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rbd_replay": "0/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_refs": "0/0", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_reserver": "1/1", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_access": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_datacache": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_dbstore": "1/5", 2024-05-08T17:16:40.741 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_flight": "1/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_lifecycle": "1/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rgw_sync": "1/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_rocksdb": "4/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_backref": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_cache": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_cleaner": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_device": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_epm": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_fixedkv_tree": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_journal": "0/5", 2024-05-08T17:16:40.742 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_lba": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_odata": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_omap": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_onode": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_t": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_seastore_tm": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_striper": "0/1", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_test": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_throttle": "1/1", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_timer": "0/1", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_tp": "0/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "debug_trace": "1/5", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "defer_client_eviction_on_laggy_osds": "false", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "device_failure_prediction_mode": "none", 2024-05-08T17:16:40.743 INFO:teuthology.orchestra.run.smithi154.stdout: "enable_experimental_unrecoverable_data_corrupting_features": "", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_dir": "/usr/lib64/ceph/erasure-code", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "err_to_graylog": "false", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "err_to_journald": "false", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "err_to_stderr": "true", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "err_to_syslog": "false", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event_tracing": "false", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_addr": "0.0.0.0", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_http_port": "9926", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_prio_limit": "5", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_sock_dir": "/var/run/ceph/", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_sort_metrics": "true", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "exporter_stats_period": "5", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "fake_statfs_for_testing": "0", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "fatal_signal_handlers": "true", 2024-05-08T17:16:40.744 INFO:teuthology.orchestra.run.smithi154.stdout: "filer_max_purge_ops": "10", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filer_max_truncate_ops": "128", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_apply_finisher_threads": "1", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_blackhole": "false", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_btrfs_clone_range": "true", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_btrfs_snap": "true", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_caller_concurrency": "10", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_collect_device_partition_information": "true", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_commit_timeout": "600.000000", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_debug_inject_read_err": "false", 2024-05-08T17:16:40.745 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_debug_omap_check": "false", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_debug_random_read_err": "0.000000", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_debug_verify_split": "false", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_dump_file": "", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_expected_throughput_bytes": "209715200.000000", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_expected_throughput_ops": "200.000000", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fadvise": "true", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fail_eio": "true", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fd_cache_shards": "16", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fd_cache_size": "128", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fiemap": "false", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fiemap_threshold": "4096", 2024-05-08T17:16:40.746 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_fsync_flushes_journal_data": "false", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_index_retry_probability": "0.000000", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_inject_stall": "0", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_journal_parallel": "false", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_journal_trailing": "false", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_journal_writeahead": "false", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_kill_at": "0", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_alloc_hint_size": "1048576", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattr_size": "0", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattr_size_btrfs": "2048", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattr_size_other": "512", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattr_size_xfs": "65536", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattrs": "0", 2024-05-08T17:16:40.747 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattrs_btrfs": "10", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattrs_other": "2", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_inline_xattrs_xfs": "10", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_sync_interval": "5.000000", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_xattr_value_size": "0", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_xattr_value_size_btrfs": "65536", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_xattr_value_size_other": "1024", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_max_xattr_value_size_xfs": "65536", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_merge_threshold": "-10", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_min_sync_interval": "0.010000", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_odsync_write": "false", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_omap_backend": "rocksdb", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_omap_backend_path": "", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_omap_header_cache_size": "1024", 2024-05-08T17:16:40.748 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_ondisk_finisher_threads": "1", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_op_thread_suicide_timeout": "180", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_op_thread_timeout": "60", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_op_threads": "2", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_punch_hole": "false", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_high_delay_multiple": "0.000000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_high_delay_multiple_bytes": "0.000000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_high_delay_multiple_ops": "0.000000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_high_threshhold": "0.900000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_low_threshhold": "0.300000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_max_bytes": "104857600", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_max_delay_multiple": "0.000000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_max_delay_multiple_bytes": "0.000000", 2024-05-08T17:16:40.749 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_max_delay_multiple_ops": "0.000000", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_queue_max_ops": "50", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_rocksdb_options": "max_background_jobs=10,compaction_readahead_size=2097152,compression=kNoCompression", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_seek_data_hole": "false", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_sloppy_crc": "false", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_sloppy_crc_block_size": "65536", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_splice": "false", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_split_multiple": "2", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_split_rand_factor": "20", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_update_to": "1000", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_bytes_hard_limit": "419430400", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_bytes_start_flusher": "41943040", 2024-05-08T17:16:40.750 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_inodes_hard_limit": "5000", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_inodes_start_flusher": "500", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_ios_hard_limit": "5000", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_btrfs_ios_start_flusher": "500", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_enable": "true", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_bytes_hard_limit": "419430400", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_bytes_start_flusher": "41943040", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_inodes_hard_limit": "5000", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_inodes_start_flusher": "500", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_ios_hard_limit": "5000", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_wbthrottle_xfs_ios_start_flusher": "500", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_xfs_extsize": "false", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "filestore_zfs_snap": "false", 2024-05-08T17:16:40.751 INFO:teuthology.orchestra.run.smithi154.stdout: "fio_dir": "/tmp/fio", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_allow_other": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_atomic_o_trunc": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_big_writes": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_debug": "false", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_default_permissions": "false", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_disable_pagecache": "false", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_max_write": "0", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_multithreaded": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_require_active_mds": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_set_user_groups": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_splice_move": "true", 2024-05-08T17:16:40.752 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_splice_read": "true", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_splice_write": "true", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_syncfs_on_mksnap": "true", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "fuse_use_invalidate_cb": "true", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "gss_ktab_client_file": "/var/lib/ceph/mon.a/gss_client_mon.a.ktab", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "gss_target_name": "ceph", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_file": "", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_inject_failure": "0", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_interval": "5", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "host": "smithi154", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_client_dedicated_thread_num": "2", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_max_inflight_ops": "128", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_max_size": "1073741824", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_path": "/tmp/ceph_immutable_object_cache", 2024-05-08T17:16:40.753 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_bps_burst": "0", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_bps_burst_seconds": "1", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_bps_limit": "0", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_iops_burst": "0", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_iops_burst_seconds": "1", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_iops_limit": "0", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_qos_schedule_tick_min": "50", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_sock": "/var/run/ceph/immutable_object_cache_sock", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "immutable_object_cache_watermark": "0.900000", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "inject_early_sigterm": "false", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "jaeger_agent_port": "6799", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "jaeger_tracing_enable": "false", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_aio": "true", 2024-05-08T17:16:40.754 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_align_min_size": "65536", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_block_align": "true", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_block_size": "4096", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_dio": "true", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_discard": "false", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_force_aio": "false", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_ignore_corruption": "false", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_max_write_bytes": "10485760", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_max_write_entries": "100", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_replay_from": "0", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_throttle_high_multiple": "0.000000", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_throttle_high_threshhold": "0.900000", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_throttle_low_threshhold": "0.600000", 2024-05-08T17:16:40.755 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_throttle_max_multiple": "0.000000", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_write_header_frequency": "0", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "journal_zero_on_create": "false", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "journaler_prefetch_periods": "10", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "journaler_prezero_periods": "5", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "journaler_write_head_interval": "15", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "key": "", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "keyfile": "", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "keyring": "/var/lib/ceph/mon/ceph-a/keyring", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_backend": "rocksdb", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_default_stripe_size": "65536", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_fsck_on_mount": "false", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_fsck_on_mount_deep": "true", 2024-05-08T17:16:40.756 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_max_bytes": "67108864", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_max_ops": "512", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_nid_prealloc": "1024", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_onode_map_size": "1024", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_rocksdb_options": "compression=kNoCompression", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_sync_submit_transaction": "false", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "kstore_sync_transaction": "false", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "librados_thread_count": "2", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "lockdep": "true", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "lockdep_force_backtrace": "false", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_coarse_timestamps": "true", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_file": "/var/log/ceph/ceph-mon.a.log", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_flush_on_exit": "false", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_graylog_host": "127.0.0.1", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_graylog_port": "12201", 2024-05-08T17:16:40.757 INFO:teuthology.orchestra.run.smithi154.stdout: "log_max_new": "1000", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_max_recent": "10000", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_stderr_prefix": "", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_stop_at_utilization": "0.970000", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_to_file": "true", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_to_graylog": "false", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_to_journald": "false", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_to_stderr": "false", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "log_to_syslog": "false", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "max_rotating_auth_attempts": "10", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_abort_on_newly_corrupt_dentry": "true", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_action_on_write_error": "1", 2024-05-08T17:16:40.758 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_alternate_name_max": "8192", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_asio_thread_count": "2", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_export_pin": "true", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_fragment_dirs": "true", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_fragment_fast_factor": "1.500000", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_fragment_interval": "5", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_fragment_size_max": "100000", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_idle_threshold": "0.000000", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_interval": "10", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_max": "-1", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_max_until": "-1", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_merge_size": "50", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_midchunk": "0.300000", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_min_rebalance": "0.100000", 2024-05-08T17:16:40.759 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_min_start": "0.200000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_minchunk": "0.001000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_mode": "0", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_need_max": "1.200000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_need_min": "0.800000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_overload_epochs": "2", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_replicate_threshold": "8000.000000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_sample_interval": "3.000000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_split_bits": "3", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_split_rd": "25000.000000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_split_size": "10000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_split_wr": "10000.000000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_target_decay": "10.000000", 2024-05-08T17:16:40.760 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_bal_unreplicate_threshold": "0.000000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_beacon_grace": "15.000000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_beacon_interval": "4.000000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_beacon_mon_down_grace": "60", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_memory_limit": "4294967296", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_mid": "0.700000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_quiesce_decay_rate": "1.000000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_quiesce_delay": "0", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_quiesce_sleep": "200", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_quiesce_splitauth": "true", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_quiesce_threshold": "524288", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_release_free_interval": "10", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_reservation": "0.050000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_trim_decay_rate": "1.000000", 2024-05-08T17:16:40.761 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_trim_interval": "1", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cache_trim_threshold": "262144", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cap_acquisition_throttle_retry_request_timeout": "0.500000", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_cap_revoke_eviction_timeout": "0.000000", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_client_delegate_inos_pct": "50", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_client_prealloc_inos": "1000", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_client_writeable_range_max_inc_objs": "1024", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_connect_bootstrapping": "false", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_damage_table_max_entries": "10000", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_data": "/var/lib/ceph/mds/ceph-a", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_debug_auth_pins": "false", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_debug_frag": "false", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_debug_scatterstat": "false", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_debug_subtrees": "false", 2024-05-08T17:16:40.762 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_decay_halflife": "5.000000", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_default_dir_hash": "2", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_defer_session_stale": "true", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_deny_all_reconnect": "false", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dir_keys_per_op": "16384", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dir_max_commit_size": "10", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dir_max_entries": "0", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dir_prefetch": "true", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dirstat_min_interval": "1.000000", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dump_cache_after_rejoin": "false", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dump_cache_on_map": "false", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dump_cache_threshold_file": "0", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_dump_cache_threshold_formatter": "1073741824", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_early_reply": "true", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_enable_op_tracker": "true", 2024-05-08T17:16:40.763 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_enforce_unique_name": "true", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_export_ephemeral_distributed": "true", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_export_ephemeral_distributed_factor": "2.000000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_export_ephemeral_random": "true", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_export_ephemeral_random_max": "0.010000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_extraordinary_events_dump_interval": "0", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_forward_all_requests_to_auth": "false", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_freeze_tree_timeout": "30.000000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_fscrypt_last_block_max_size": "4096", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_go_bad_corrupt_dentry": "true", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_hack_allow_loading_invalid_metadata": "false", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_health_cache_threshold": "1.500000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_health_summarize_threshold": "10", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_heartbeat_grace": "15.000000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_heartbeat_reset_grace": "1000", 2024-05-08T17:16:40.764 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_health_dummy": "false", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_journal_corrupt_dentry_first": "0.000000", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_migrator_session_race": "false", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_rename_corrupt_dentry_first": "0.000000", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_skip_replaying_inotable": "false", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_inject_traceless_reply_probability": "0.000000", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_join_fs": "", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_journal_format": "1", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_create_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_export_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_import_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_journal_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_journal_expire_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_journal_replay_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_link_at": "0", 2024-05-08T17:16:40.765 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_mdstable_at": "0", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_openc_at": "0", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_rename_at": "0", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_shutdown_at": "0", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_kill_skip_replaying_inotable": "false", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_event_large_threshold": "524288", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_events_per_segment": "1024", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_major_segment_event_ratio": "12", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_max_events": "-1", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_max_segments": "128", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_pause": "false", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_segment_size": "0", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_skip_corrupt_events": "false", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_skip_unbounded_events": "false", 2024-05-08T17:16:40.766 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_trim_decay_rate": "1.000000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_trim_threshold": "128", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_trim_upkeep_interval": "1000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_log_warn_factor": "2.000000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_caps_per_client": "1048576", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_completed_flushes": "100000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_completed_requests": "100000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_export_size": "20971520", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_file_recover": "32", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_purge_files": "64", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_purge_ops": "8192", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_purge_ops_per_pg": "0.500000", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_scrub_ops_in_progress": "5", 2024-05-08T17:16:40.767 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_max_snaps_per_dir": "100", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_metrics_update_interval": "2", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_min_caps_per_client": "100", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_min_caps_working_set": "10000", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_mon_shutdown_timeout": "5.000000", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_numa_node": "-1", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_oft_prefetch_dirfrags": "false", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_complaint_time": "30.000000", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_history_duration": "600", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_history_size": "20", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_history_slow_op_size": "20", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_history_slow_op_threshold": "10", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_op_log_threshold": "5", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_ping_grace": "15", 2024-05-08T17:16:40.768 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_ping_interval": "5", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_purge_queue_busy_flush_period": "1.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_global_max_decay_threshold": "131072", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_max_caps": "30000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_max_decay_rate": "1.500000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_max_decay_threshold": "131072", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_warning_decay_rate": "60.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_recall_warning_threshold": "262144", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_reconnect_timeout": "45.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_replay_interval": "1.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_replay_unsafe_with_closed_session": "false", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_request_load_average_decay_rate": "60.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_root_ino_gid": "0", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_root_ino_uid": "0", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_scatter_nudge_interval": "5.000000", 2024-05-08T17:16:40.769 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_blocklist_on_evict": "true", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_blocklist_on_timeout": "true", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_cache_liveness_decay_rate": "300.000000", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_cache_liveness_magnitude": "10", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_cap_acquisition_decay_rate": "30.000000", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_cap_acquisition_throttle": "100000", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_max_caps_throttle_ratio": "1.100000", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_session_metadata_threshold": "16777216", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sessionmap_keys_per_op": "1024", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_shutdown_check": "0", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_skip_ino": "0", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_sleep_rank_change": "0.000000", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_snap_max_uid": "4294967294", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_snap_min_uid": "0", 2024-05-08T17:16:40.770 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_snap_rstat": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_standby_replay_damaged": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_symlink_recovery": "true", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_task_status_update_interval": "2.000000", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_thrash_exports": "0", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_thrash_fragments": "0", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_tick_interval": "5.000000", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_valgrind_exit": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_verify_backtrace": "1", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_verify_scatter": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_wipe_ino_prealloc": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mds_wipe_sessions": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "mempool_debug": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "memstore_debug_omit_block_device_write": "false", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "memstore_device_bytes": "1073741824", 2024-05-08T17:16:40.771 INFO:teuthology.orchestra.run.smithi154.stdout: "memstore_page_set": "false", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "memstore_page_size": "65536", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_client_bytes": "134217728", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_client_messages": "512", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_client_service_daemon_unregister_timeout": "1.000000", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_connect_retry_interval": "1.000000", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_data": "/var/lib/ceph/mgr/ceph-a", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_debug_aggressive_pg_num_changes": "false", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_disabled_modules": "", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_initial_modules": "restful iostat nfs", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_max_pg_num_change": "128", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_mds_bytes": "134217728", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_mds_messages": "128", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_module_path": "/usr/share/ceph/mgr", 2024-05-08T17:16:40.772 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_mon_bytes": "134217728", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_mon_messages": "128", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_osd_bytes": "536870912", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_osd_messages": "8192", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_pool": "true", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_service_beacon_grace": "60.000000", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_standby_modules": "true", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_stats_period": "5", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_stats_threshold": "5", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_tick_period": "2", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr_ttl_cache_expire_seconds": "0", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_accept_timeout_factor": "2.000000", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_allow_pool_delete": "true", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_allow_pool_size_one": "true", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_auth_validate_all_caps": "true", 2024-05-08T17:16:40.773 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cache_target_full_warn_ratio": "0.660000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_clean_pg_upmaps_per_chunk": "256", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_bytes": "104857600", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_directed_command_retry": "5", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_hunt_interval": "3.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_hunt_interval_backoff": "1.500000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_hunt_interval_max_multiple": "10.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_hunt_interval_min_multiple": "1.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_hunt_parallel": "3", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_log_interval": "1.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_max_log_entries_per_message": "1000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_ping_interval": "10.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_ping_timeout": "30.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_client_target_rank": "-1", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_clock_drift_allowed": "1.000000", 2024-05-08T17:16:40.774 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_clock_drift_warn_backoff": "5.000000", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_file": "default=/var/log/ceph/ceph.$channel.log cluster=/var/log/ceph/ceph.log", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_file_level": "debug", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_file": "true", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_graylog": "false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_graylog_host": "127.0.0.1", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_graylog_port": "12201", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_journald": "false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_stderr": "false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_syslog": "default=false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_syslog_facility": "daemon", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cluster_log_to_syslog_level": "info", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_compact_on_bootstrap": "false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_compact_on_start": "false", 2024-05-08T17:16:40.775 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_compact_on_trim": "true", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_con_tracker_persist_interval": "10", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_con_tracker_score_halflife": "43200", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_config_key_max_entry_size": "65536", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_cpu_threads": "4", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_crush_min_required_version": "hammer", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_daemon_bytes": "419430400", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_data": "/var/lib/ceph/mon/ceph-a", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_data_avail_crit": "5", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_data_avail_warn": "5", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_data_size_warn": "16106127360", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_block_osdmap_trim": "false", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_deprecated_as_obsolete": "false", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_dump_json": "false", 2024-05-08T17:16:40.776 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_dump_location": "/var/log/ceph/ceph-mon.a.tdump", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_dump_transactions": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_extra_checks": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_no_initial_persistent_features": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_no_require_bluestore_for_ec_overwrites": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_no_require_reef": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_no_require_squid": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_debug_unsafe_allow_tier_with_nonempty_snaps": "false", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_delta_reset_interval": "10.000000", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_dns_srv_name": "ceph-mon", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_down_mkfs_grace": "120", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_down_uptime_grace": "60", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_election_default_strategy": "3", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_election_timeout": "5.000000", 2024-05-08T17:16:40.777 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_elector_ignore_propose_margin": "0.000500", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_elector_ping_divisor": "2", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_elector_ping_timeout": "2.000000", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_enable_op_tracker": "true", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_fake_pool_delete": "false", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_force_quorum_join": "false", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_globalid_prealloc": "10000", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_detail_to_clog": "false", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_log_update_period": "5", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_max_detail": "50", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_to_clog": "true", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_to_clog_interval": "600", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_health_to_clog_tick_interval": "60.000000", 2024-05-08T17:16:40.778 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_host": "172.21.15.154,[v2:172.21.15.154:3301,v1:172.21.15.154:6790],172.21.15.183", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_host_override": "", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_initial_members": "", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_inject_pg_merge_bounce_probability": "0.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_inject_sync_get_chunk_delay": "0.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_inject_transaction_delay_max": "10.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_inject_transaction_delay_probability": "0.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_keyvaluedb": "rocksdb", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_lease": "5.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_lease_ack_timeout_factor": "2.000000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_lease_renew_interval_factor": "0.600000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_log_full_interval": "50", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_log_max": "10000", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_log_max_summary": "50", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_log_entries_per_event": "4096", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_log_epochs": "500", 2024-05-08T17:16:40.779 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_mdsmap_epochs": "500", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_mgrmap_epochs": "500", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_osd": "10000", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_pg_per_osd": "10000", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_pool_pg_num": "65536", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_max_snap_prune_per_epoch": "100", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mds_blocklist_interval": "86400.000000", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mds_force_trim_to": "0", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mds_skip_sanity": "false", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_memory_autotune": "true", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_memory_target": "2147483648", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_beacon_grace": "30", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_blocklist_interval": "86400.000000", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_digest_period": "5", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_inactive_grace": "60", 2024-05-08T17:16:40.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_mkfs_grace": "240", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_mgr_proxy_client_bytes_ratio": "0.300000", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_min_osdmap_epochs": "50", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_complaint_time": "30", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_history_duration": "600", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_history_size": "20", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_history_slow_op_size": "20", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_history_slow_op_threshold": "10", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_op_log_threshold": "5", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_adjust_down_out_interval": "true", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_adjust_heartbeat_grace": "true", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_auto_mark_auto_out_in": "true", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_auto_mark_in": "false", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_auto_mark_new_in": "true", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_backfillfull_ratio": "0.900000", 2024-05-08T17:16:40.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_blocklist_default_expire": "3600.000000", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_cache_size": "500", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_cache_size_min": "134217728", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_crush_smoke_test": "true", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_destroyed_out_interval": "600", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_down_out_interval": "600", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_down_out_subtree_limit": "rack", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_err_op_age_ratio": "128.000000", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_force_trim_to": "0", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_full_ratio": "0.950000", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_initial_require_min_compat_client": "luminous", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_laggy_halflife": "3600", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_laggy_max_interval": "300", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_laggy_weight": "0.300000", 2024-05-08T17:16:40.782 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_mapping_pgs_per_chunk": "4096", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_max_creating_pgs": "1024", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_max_initial_pgs": "1024", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_min_down_reporters": "2", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_min_in_ratio": "0.750000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_min_up_ratio": "0.300000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_nearfull_ratio": "0.850000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_prime_pg_temp": "true", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_prime_pg_temp_max_estimate": "0.250000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_prime_pg_temp_max_time": "0.500000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_report_timeout": "900", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_reporter_subtree_level": "osd", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_snap_trim_queue_warn_on": "32768", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_warn_num_repaired": "10", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osd_warn_op_age": "32.000000", 2024-05-08T17:16:40.783 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osdmap_full_prune_enabled": "true", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osdmap_full_prune_interval": "2", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osdmap_full_prune_min": "15", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_osdmap_full_prune_txsize": "2", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_check_down_all_threshold": "0.500000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_stuck_threshold": "60", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_warn_max_object_skew": "0.000000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_warn_min_objects": "10000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_warn_min_per_osd": "0", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pg_warn_min_pool_objects": "1000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pool_quota_crit_threshold": "0", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_pool_quota_warn_threshold": "0", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_probe_timeout": "2.000000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_reweight_max_change": "0.050000", 2024-05-08T17:16:40.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_reweight_max_osds": "4", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_reweight_min_bytes_per_osd": "10", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_reweight_min_pgs_per_osd": "4", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_rocksdb_options": "write_buffer_size=33554432,compression=kNoCompression,level_compaction_dynamic_level_bytes=true", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_scrub_inject_crc_mismatch": "0.000000", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_scrub_inject_missing_keys": "0.000000", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_scrub_interval": "300", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_scrub_max_keys": "100", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_scrub_timeout": "300", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_session_timeout": "300", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_smart_report_timeout": "5", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_stat_smooth_intervals": "6", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_stretch_cluster_recovery_ratio": "0.600000", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_stretch_pool_min_size": "2", 2024-05-08T17:16:40.785 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_stretch_pool_size": "4", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_stretch_recovery_min_wait": "15.000000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_subscribe_interval": "86400.000000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_debug": "false", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_max_payload_keys": "2000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_max_payload_size": "1048576", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_provider_kill_at": "0", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_requester_kill_at": "0", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_sync_timeout": "60.000000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_target_pg_per_osd": "100", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_tick_interval": "5", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_timecheck_interval": "300.000000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_timecheck_skew_interval": "30.000000", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_use_min_delay_socket": "false", 2024-05-08T17:16:40.786 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_older_version_delay": "604800", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_cache_pools_without_hit_sets": "true", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_crush_straw_calc_version_zero": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_degraded_stretch_mode": "true", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_filestore_osds": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_insecure_global_id_reclaim": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_insecure_global_id_reclaim_allowed": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_legacy_crush_tunables": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_misplaced": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_msgr2_not_enabled": "true", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_older_version": "true", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_osd_down_out_interval_zero": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_pool_no_app": "true", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_pool_no_redundancy": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_pool_pg_num_not_power_of_two": "false", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_slow_ping_ratio": "0.050000", 2024-05-08T17:16:40.787 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_slow_ping_time": "0.000000", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_on_too_few_osds": "false", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_pg_not_deep_scrubbed_ratio": "0.750000", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "mon_warn_pg_not_scrubbed_ratio": "0.500000", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "monmap": "", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_admin_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_admin_fid": "0x7200000000000001:0x0", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_ha_endpoint": "192.168.180.182@tcp:12345:1:1", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_my_endpoint": "192.168.180.182@tcp:12345:4:1", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_my_fid": "0x7200000000000001:0x0", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_profile_fid": "0x7000000000000001:0x0", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "motr_tracing_enabled": "false", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_op_threads": "3", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_buffer_size": "131072", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_cm": "false", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_device_name": "", 2024-05-08T17:16:40.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_dscp": "96", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_enable_hugepage": "false", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_gid_idx": "0", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_local_gid": "", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_polling_us": "1000", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_port_num": "1", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_receive_buffers": "32768", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_receive_queue_len": "4096", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_roce_ver": "1", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_send_buffers": "1024", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_sl": "3", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_support_srq": "true", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_rdma_type": "ib", 2024-05-08T17:16:40.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_async_reap_threshold": "5", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_before_connect": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_ipv4": "true", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_ipv6": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_msgr1": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_msgr2": "true", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_port_max": "7568", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_port_min": "6800", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_prefer_ipv4": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_retry_count": "3", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_bind_retry_delay": "5", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_blackhole_client": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_blackhole_mds": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_blackhole_mgr": "false", 2024-05-08T17:16:40.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_blackhole_mon": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_blackhole_osd": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_client_mode": "crc secure", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_cluster_mode": "crc secure", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_cluster_type": "", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_compress_secure": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_connection_idle_timeout": "900", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_connection_ready_timeout": "10", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_crc_data": "true", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_crc_header": "true", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_die_on_bad_msg": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_die_on_bug": "true", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_die_on_old_message": "true", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_die_on_skipped_message": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_die_on_unhandled_msg": "false", 2024-05-08T17:16:40.791 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dispatch_throttle_bytes": "104857600", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_coremask": "0xF", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_debug_allow_loopback": "false", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_devs_allowlist": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_enable_tso": "true", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_gateway_ipv4_addr": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_host_ipv4_addr": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_hugepages": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_hw_flow_control": "true", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_hw_queue_weight": "1.000000", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_lro": "true", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_memory_channel": "4", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_netmask_ipv4_addr": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_pmd": "", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_port_id": "0", 2024-05-08T17:16:40.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dpdk_rx_buffer_count_per_core": "8192", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dump_corrupt_message_level": "1", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_dump_on_send": "false", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_initial_backoff": "0.200000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_delay_max": "1.000000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_delay_probability": "0.000000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_delay_type": "", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_internal_delays": "0.000000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_network_congestion": "0", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_inject_socket_failures": "5000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_learn_addr_from_peer": "true", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_max_accept_failures": "4", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_max_backoff": "15.000000", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_mon_client_mode": "secure crc", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_mon_cluster_mode": "secure crc", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_mon_service_mode": "secure crc", 2024-05-08T17:16:40.793 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_osd_compress_min_size": "1024", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_osd_compress_mode": "none", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_osd_compression_algorithm": "snappy", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_pq_max_tokens_per_priority": "16777216", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_pq_min_cost": "65536", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_public_type": "", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_service_mode": "crc secure", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_tcp_listen_backlog": "512", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_tcp_nodelay": "true", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_tcp_prefetch_max_size": "4096", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_tcp_rcvbuf": "0", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "ms_type": "async", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "no_config_file": "false", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_completion_locks_per_session": "32", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_debug_inject_relock_delay": "false", 2024-05-08T17:16:40.794 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_inflight_op_bytes": "104857600", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_inflight_ops": "1024", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_inject_no_watch_ping": "false", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_retry_writes_after_first_reply": "false", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_tick_interval": "5.000000", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objecter_timeout": "10.000000", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objectstore_blackhole": "false", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "objectstore_debug_throw_on_failed_txc": "false", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "openssl_engine_opts": "", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_delay_time": "5.000000", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_hist_halflife": "1000", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_max_low_ops": "2", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_max_ops": "4", 2024-05-08T17:16:40.795 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_min_evict_effort": "0.100000", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_quantize_effort": "0.100000", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_agent_slop": "0.020000", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_aggregated_slow_ops_logging": "true", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_allow_recovery_below_min_size": "true", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_asio_thread_count": "2", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_async_recovery_min_cost": "100", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_auto_mark_unfound_lost": "false", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backfill_retry_interval": "30.000000", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backfill_scan_max": "512", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backfill_scan_min": "64", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backoff_on_degraded": "false", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backoff_on_peering": "false", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_backoff_on_unfound": "true", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_beacon_report_interval": "300", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bench_duration": "30", 2024-05-08T17:16:40.796 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bench_large_size_max_throughput": "104857600", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bench_max_block_size": "67108864", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_bench_small_size_max_iops": "100", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_blkin_trace_all": "false", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_blocked_scrub_grace_period": "120", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_calc_pg_upmaps_aggressively": "true", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_calc_pg_upmaps_aggressively_fast": "true", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_calc_pg_upmaps_local_fallback_retries": "100", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_check_for_log_corruption": "false", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_check_max_object_name_len_on_startup": "true", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_class_default_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_class_dir": "/usr/lib64/rados-classes", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_class_load_list": "cephfs hello journal lock log numops otp rbd refcount rgw rgw_gc timeindex user version cas cmpomap queue 2pc_queue fifo", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_class_update_on_start": "true", 2024-05-08T17:16:40.797 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_client_message_cap": "256", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_client_message_size_cap": "524288000", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_client_op_priority": "63", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_client_watch_timeout": "30", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_command_max_records": "256", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_compact_on_start": "false", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_copyfrom_max_chunk": "8388608", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_crush_chooseleaf_type": "0", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_crush_initial_weight": "-1.000000", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_crush_update_on_start": "true", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_crush_update_weight_set": "true", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_data": "/var/lib/ceph/osd/ceph-a", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_crash_on_ignored_backoff": "false", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_deep_scrub_sleep": "0.000000", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_drop_ping_duration": "0", 2024-05-08T17:16:40.798 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_drop_ping_probability": "0.000000", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_feed_pullee": "-1", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_inject_copyfrom_error": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_inject_dispatch_delay_duration": "0.100000", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_inject_dispatch_delay_probability": "0.000000", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_misdirected_ops": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_no_acting_change": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_no_purge_strays": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_op_order": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_pg_log_writeout": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_pretend_recovery_active": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_random_push_read_error": "0.000000", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_reject_backfill_probability": "0.000000", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_shutdown": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_skip_full_check_in_backfill_reservation": "false", 2024-05-08T17:16:40.799 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_skip_full_check_in_recovery": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_trim_objects": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_verify_cached_snaps": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_verify_missing_on_start": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_verify_snaps": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_debug_verify_stray_on_activate": "false", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_interval": "604800.000000", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_keys": "1024", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_large_omap_object_key_threshold": "200000", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_large_omap_object_value_sum_threshold": "1073741824", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_randomize_ratio": "0.150000", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_stride": "524288", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_deep_scrub_update_digest_min_age": "7200", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_default_data_pool_replay_window": "5", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_default_notify_timeout": "30", 2024-05-08T17:16:40.800 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_delete_sleep": "0.000000", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_delete_sleep_hdd": "5.000000", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_delete_sleep_hybrid": "1.000000", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_delete_sleep_ssd": "1.000000", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_discard_disconnected_ops": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_enable_op_tracker": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_erasure_code_plugins": "jerasure lrc", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_extblkdev_plugins": "vdo", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_failsafe_full_ratio": "0.970000", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_fast_fail_on_connection_refused": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_fast_info": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_fast_shutdown": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_fast_shutdown_notify_mon": "true", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_fast_shutdown_timeout": "15", 2024-05-08T17:16:40.801 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_find_best_info_ignore_history_les": "false", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_force_auth_primary_missing_objects": "100", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_force_recovery_pg_log_entries_factor": "1.300000", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_function_tracing": "false", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_grace": "20", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_interval": "6", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_min_healthy_ratio": "0.330000", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_min_peers": "10", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_min_size": "2000", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_stale": "600", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_heartbeat_use_min_delay_socket": "false", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_hit_set_max_size": "100000", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_hit_set_min_size": "1000", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_hit_set_namespace": ".ceph-internal", 2024-05-08T17:16:40.802 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_ignore_stale_divergent_priors": "false", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_inject_bad_map_crc_probability": "0.000000", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_inject_failure_on_pg_removal": "false", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_journal": "/var/lib/ceph/osd/ceph-a/journal", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_journal_flush_on_shutdown": "true", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_journal_size": "5120", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_kill_backfill_at": "0", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_loop_before_reset_tphandle": "64", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_map_cache_size": "50", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_map_dedup": "true", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_map_message_max": "40", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_map_message_max_bytes": "10485760", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_map_share_max_epochs": "40", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_attr_name_len": "100", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_attr_size": "0", 2024-05-08T17:16:40.803 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_backfills": "1", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_markdown_count": "5", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_markdown_period": "600", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_object_name_len": "2048", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_object_namespace_len": "256", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_object_size": "134217728", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_omap_bytes_per_request": "1073741824", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_omap_entries_per_request": "1024", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_pg_blocked_by": "16", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_pg_log_entries": "10000", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_pg_per_osd_hard_ratio": "3.000000", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_pgls": "1024", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_push_cost": "8388608", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_push_objects": "10", 2024-05-08T17:16:40.804 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_scrubs": "3", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_snap_prune_intervals_per_epoch": "512", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_trimming_pgs": "2", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_write_op_reply_len": "64", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_max_write_size": "90", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_force_run_benchmark_on_init": "false", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_iops_capacity_threshold_hdd": "500.000000", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_iops_capacity_threshold_ssd": "80000.000000", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_max_capacity_iops_hdd": "315.000000", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_max_capacity_iops_ssd": "21500.000000", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_max_sequential_bandwidth_hdd": "157286400", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_max_sequential_bandwidth_ssd": "1258291200", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_override_recovery_settings": "false", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_profile": "balanced", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_anticipation_timeout": "0.000000", 2024-05-08T17:16:40.805 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_best_effort_lim": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_best_effort_res": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_best_effort_wgt": "1", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_recovery_lim": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_recovery_res": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_background_recovery_wgt": "1", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_client_lim": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_client_res": "0.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_scheduler_client_wgt": "1", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mclock_skip_benchmark": "false", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_base": "805306368", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_cache_min": "134217728", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_cache_resize_interval": "1.000000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_expected_fragmentation": "0.150000", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_target": "4294967296", 2024-05-08T17:16:40.806 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_target_autotune": "false", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_memory_target_cgroup_limit_ratio": "0.800000", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_min_pg_log_entries": "250", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_min_recovery_priority": "0", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mon_heartbeat_interval": "30", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mon_heartbeat_stat_stale": "3600", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mon_report_interval": "5", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mon_report_max_in_flight": "2", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_mon_shutdown_timeout": "5.000000", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_num_cache_shards": "32", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_num_op_tracker_shard": "32", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_numa_auto_affinity": "true", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_numa_node": "-1", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_numa_prefer_iface": "true", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_object_clean_region_max_num_intervals": "10", 2024-05-08T17:16:40.807 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_objecter_finishers": "1", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_objectstore": "bluestore", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_objectstore_fuse": "false", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_objectstore_tracing": "false", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_complaint_time": "30.000000", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_history_duration": "600", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_history_size": "20", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_history_slow_op_size": "20", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_history_slow_op_threshold": "10.000000", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_log_threshold": "5", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_shards": "0", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_shards_hdd": "5", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_shards_ssd": "8", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_threads_per_shard": "0", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_threads_per_shard_hdd": "1", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_num_threads_per_shard_ssd": "2", 2024-05-08T17:16:40.808 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_pq_max_tokens_per_priority": "4194304", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_pq_min_cost": "65536", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_queue": "mclock_scheduler", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_queue_cut_off": "high", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_thread_suicide_timeout": "150", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_op_thread_timeout": "15", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_open_classes_on_start": "true", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_os_flags": "0", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_peering_op_priority": "255", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_delete_cost": "1048576", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_delete_priority": "5", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_epoch_max_lag_factor": "2.000000", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_epoch_persisted_max_stale": "40", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_log_dups_tracked": "3000", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_log_trim_max": "10000", 2024-05-08T17:16:40.809 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_log_trim_min": "100", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_max_concurrent_snap_trims": "2", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_object_context_cache_count": "64", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_stat_report_interval_max_epochs": "500", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pg_stat_report_interval_max_seconds": "5", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_max_evict_check_size": "10", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_min_evict_age": "0", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_min_flush_age": "0", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_target_dirty_high_ratio": "0.600000", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_target_dirty_ratio": "0.400000", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_cache_target_full_ratio": "0.800000", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_crimson": "false", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_crush_rule": "-1", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_ec_fast_read": "false", 2024-05-08T17:16:40.810 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_erasure_code_profile": "plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flag_bulk": "false", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flag_hashpspool": "true", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flag_nodelete": "false", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flag_nopgchange": "false", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flag_nosizechange": "false", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_flags": "0", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_hit_set_bloom_fpp": "0.050000", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_min_size": "0", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_pg_autoscale_mode": "off", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_pg_num": "32", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_pgp_num": "0", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_read_lease_ratio": "0.800000", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_read_ratio": "70", 2024-05-08T17:16:40.811 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_size": "2", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_default_type": "replicated", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_erasure_code_stripe_unit": "4096", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_pool_use_gmt_hitset": "true", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_push_per_object_cost": "1000", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_read_ec_check_for_errors": "false", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recover_clone_overlap": "true", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recover_clone_overlap_limit": "10", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_cost": "20971520", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_delay_start": "0.000000", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_active": "0", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_active_hdd": "3", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_active_ssd": "10", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_chunk": "8388608", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_omap_entries_per_chunk": "8096", 2024-05-08T17:16:40.812 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_max_single_start": "1", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_op_priority": "3", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_op_warn_multiple": "16", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_priority": "5", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_retry_interval": "30.000000", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_sleep": "0.000000", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_sleep_hdd": "0.100000", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_sleep_hybrid": "0.025000", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_recovery_sleep_ssd": "0.000000", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_repair_during_recovery": "false", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_requested_scrub_priority": "120", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_rocksdb_iterator_bounds_enabled": "true", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_rollback_to_cluster_snap": "", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_auto_repair": "false", 2024-05-08T17:16:40.813 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_auto_repair_num_errors": "5", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_backoff_ratio": "0.660000", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_begin_hour": "0", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_begin_week_day": "0", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_chunk_max": "25", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_chunk_min": "5", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_cost": "52428800", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_disable_reservation_queuing": "false", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_during_recovery": "false", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_end_hour": "0", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_end_week_day": "0", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_event_cost": "4096", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_extended_sleep": "0.000000", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_interval_randomize_ratio": "0.500000", 2024-05-08T17:16:40.814 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_invalid_stats": "true", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_load_threshold": "0.500000", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_max_interval": "604800.000000", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_max_preemptions": "5", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_min_interval": "86400.000000", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_priority": "5", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_scrub_sleep": "0.000000", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_shallow_scrub_chunk_max": "100", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_shallow_scrub_chunk_min": "50", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_shutdown_pgref_assert": "false", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_skip_data_digest": "false", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_smart_report_timeout": "5", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_cost": "1048576", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_priority": "5", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_sleep": "0.000000", 2024-05-08T17:16:40.815 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_sleep_hdd": "5.000000", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_sleep_hybrid": "2.000000", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_snap_trim_sleep_ssd": "0.000000", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_stats_update_period_not_scrubbing": "120", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_stats_update_period_scrubbing": "15", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_target_pg_log_entries_per_osd": "300000", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_target_transaction_size": "30", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_hit_set_count": "4", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_hit_set_grade_decay_rate": "20", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_hit_set_period": "1200", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_hit_set_search_last_n": "1", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_hit_set_type": "bloom", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_min_read_recency_for_promote": "1", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_min_write_recency_for_promote": "1", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_default_cache_mode": "writeback", 2024-05-08T17:16:40.816 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_promote_max_bytes_sec": "5242880", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tier_promote_max_objects_sec": "25", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_tracing": "false", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_use_stale_snap": "false", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_uuid": "00000000-0000-0000-0000-000000000000", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "osdc_blkin_trace_all": "false", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_kill_at": "0", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_max_join_drift": "10", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_min": "500", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_min_wait": "0.050000", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_propose_interval": "1.000000", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_service_trim_max": "500", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_service_trim_max_multiplier": "20", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_service_trim_min": "10", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_stash_full_interval": "25", 2024-05-08T17:16:40.817 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_trim_max": "500", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "paxos_trim_min": "250", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "perf": "true", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "pid_file": "/var/run/ceph/ceph-mon.a.pid", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin_crypto_accelerator": "crypto_isal", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin_dir": "/usr/lib64/ceph", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "-", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrv": "", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "public_bind_addr": "-", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "public_network": "", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "public_network_interface": "", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "qat_compressor_enabled": "false", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "qat_compressor_session_max_number": "256", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "rados_mon_op_timeout": "0", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "rados_osd_op_timeout": "0", 2024-05-08T17:16:40.818 INFO:teuthology.orchestra.run.smithi154.stdout: "rados_tracing": "false", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_atime_update_interval": "60", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_auto_exclusive_lock_until_manual_request": "true", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_balance_parent_reads": "false", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_balance_snap_reads": "false", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_blkin_trace_all": "false", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_blocklist_expire_seconds": "0", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_blocklist_on_break_lock": "true", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache": "true", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_block_writes_upfront": "false", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_max_dirty": "25165824", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_max_dirty_age": "1.000000", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_max_dirty_object": "0", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_policy": "writearound", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_size": "33554432", 2024-05-08T17:16:40.819 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_target_dirty": "16777216", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_cache_writethrough_until_flush": "true", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_clone_copy_on_read": "false", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_compression_hint": "none", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_concurrent_management_ops": "10", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_config_pool_override_update_timestamp": "0", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_clone_format": "auto", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_data_pool": "", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_features": "61", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_format": "2", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_map_options": "", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_order": "22", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_pool": "rbd", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_snapshot_quiesce_mode": "required", 2024-05-08T17:16:40.820 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_stripe_count": "0", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_default_stripe_unit": "0", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_disable_zero_copy_writes": "true", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_discard_granularity_bytes": "65536", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_discard_on_zeroed_write_same": "true", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_enable_alloc_hint": "true", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_invalidate_object_map_on_timeout": "true", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_io_scheduler": "simple", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_io_scheduler_simple_max_delay": "0", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_commit_age": "5.000000", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_max_concurrent_object_sets": "0", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_max_payload_bytes": "16384", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_object_flush_age": "0.000000", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_object_flush_bytes": "1048576", 2024-05-08T17:16:40.821 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_object_flush_interval": "0", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_object_max_in_flight_appends": "0", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_object_writethrough_until_flush": "true", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_order": "24", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_pool": "", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_journal_splay_width": "4", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_localize_parent_reads": "false", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_localize_snap_reads": "false", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_concurrent_image_deletions": "1", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_concurrent_image_syncs": "5", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_delete_retry_interval": "30.000000", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_perf_stats_prio": "5", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_policy_migration_throttle": "300", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_policy_rebalance_timeout": "0.000000", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_policy_type": "simple", 2024-05-08T17:16:40.822 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_policy_update_throttle_interval": "1.000000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_image_state_check_interval": "30", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_journal_commit_age": "5.000000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_journal_poll_age": "5.000000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_leader_heartbeat_interval": "5", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_leader_max_acquire_attempts_before_break": "3", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_leader_max_missed_heartbeats": "2", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_autotune": "true", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_base": "805306368", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_cache_autotune_interval": "30.000000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_cache_min": "134217728", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_cache_resize_interval": "5.000000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_expected_fragmentation": "0.150000", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_memory_target": "4294967296", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_perf_stats_prio": "5", 2024-05-08T17:16:40.823 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_pool_replayers_refresh_interval": "30", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirror_sync_point_update_age": "30.000000", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirroring_delete_delay": "0", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirroring_max_mirroring_snapshots": "5", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirroring_replay_delay": "0", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mirroring_resync_after_disconnect": "false", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_move_parent_to_trash_on_remove": "false", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_move_to_trash_on_remove": "false", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_move_to_trash_on_remove_expire_seconds": "0", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_mtime_update_interval": "60", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_non_blocking_aio": "true", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_op_thread_timeout": "60", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_op_threads": "1", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_parent_cache_enabled": "false", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_persistent_cache_mode": "disabled", 2024-05-08T17:16:40.824 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_persistent_cache_path": "/tmp", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_persistent_cache_size": "1073741824", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_plugins": "", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_bps_burst": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_bps_burst_seconds": "1", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_bps_limit": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_exclude_ops": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_iops_burst": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_iops_burst_seconds": "1", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_iops_limit": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_bps_burst": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_bps_burst_seconds": "1", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_bps_limit": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_iops_burst": "0", 2024-05-08T17:16:40.825 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_iops_burst_seconds": "1", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_read_iops_limit": "0", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_schedule_tick_min": "50", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_bps_burst": "0", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_bps_burst_seconds": "1", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_bps_limit": "0", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_iops_burst": "0", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_iops_burst_seconds": "1", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_qos_write_iops_limit": "0", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_quiesce_notification_attempts": "10", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_read_from_replica_policy": "default", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_readahead_disable_after_bytes": "52428800", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_readahead_max_bytes": "524288", 2024-05-08T17:16:40.826 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_readahead_trigger_requests": "10", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_request_timed_out_seconds": "30", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_skip_partial_discard": "true", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_sparse_read_threshold_bytes": "65536", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_tracing": "false", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_validate_names": "true", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd_validate_pool": "true", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "restapi_base_url": "", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "restapi_log_level": "", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_account_default_quota_max_objects": "-1", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_account_default_quota_max_size": "-1", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_acl_grants_max_num": "100", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_admin_entry": "admin", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_allow_notification_secrets_in_cleartext": "false", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_backend_store": "rados", 2024-05-08T17:16:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_barbican_url": "", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_beast_enable_async": "true", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_counters_cache": "false", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_counters_cache_size": "10000", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_default_quota_max_objects": "-1", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_default_quota_max_size": "-1", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_index_max_aio": "128", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_index_transaction_instrumentation": "false", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_quota_cache_size": "10000", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_quota_ttl": "600", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_bucket_sync_spawn_window": "20", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_cache_enabled": "true", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_cache_expiry_interval": "900", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_cache_lru_size": "25000", 2024-05-08T17:16:40.828 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_config_store": "rados", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_content_length_compat": "false", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_copy_obj_progress": "true", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_copy_obj_progress_every_bytes": "1048576", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_cors_rules_max_num": "100", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_cross_domain_policy": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_default_encryption_key": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_addr": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_ca_path": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_client_cert": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_client_key": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_kms_key_template": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_password": "", 2024-05-08T17:16:40.829 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_s3_key_template": "$keyid", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_kmip_username": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_require_ssl": "true", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_s3_kms_backend": "barbican", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_s3_kms_encryption_keys": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_backend": "vault", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_key_template": "%bucket_id", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_addr": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_auth": "token", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_namespace": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_prefix": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_secret_engine": "transit", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_ssl_cacert": "", 2024-05-08T17:16:40.830 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_ssl_clientcert": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_ssl_clientkey": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_token_file": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_sse_s3_vault_verify_ssl": "true", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_suppress_logs": "true", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_addr": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_auth": "token", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_namespace": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_prefix": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_secret_engine": "transit", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_ssl_cacert": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_ssl_clientcert": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_ssl_clientkey": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_token_file": "", 2024-05-08T17:16:40.831 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_crypt_vault_verify_ssl": "true", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_curl_buffersize": "524288", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_curl_low_speed_limit": "1024", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_curl_low_speed_time": "300", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_curl_tcp_keepalive": "0", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_curl_wait_timeout_ms": "1000", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_datacache_persistent_path": "/tmp/rgw_datacache/", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_datacache_size": "1073741824", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_evict_cache_on_start": "true", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_eviction_policy": "lru", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_fadvise": "4", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_l1_local_datacache_enabled": "false", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_libaio_aio_num": "64", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d3n_libaio_aio_threads": "20", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d4n_host": "127.0.0.1", 2024-05-08T17:16:40.832 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_d4n_port": "6379", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data": "/var/lib/ceph/radosgw/ceph-a", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_log_changes_size": "1000", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_log_num_shards": "128", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_log_obj_prefix": "data_log", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_log_window": "30", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_notify_interval_msec": "0", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_sync_poll_interval": "20", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_data_sync_spawn_window": "20", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_debug_inject_latency_bi_unlink": "0", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_debug_inject_olh_cancel_modification_err": "false", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_debug_inject_set_olh_err": "0", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_default_data_log_backing": "fifo", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_default_realm_info_oid": "default.realm", 2024-05-08T17:16:40.833 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_default_region_info_oid": "default.region", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_default_zone_info_oid": "default.zone", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_default_zonegroup_info_oid": "default.zonegroup", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_defer_to_bucket_acls": "", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_delete_multi_obj_max_num": "1000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_admin_lim": "0.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_admin_res": "100.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_admin_wgt": "100.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_auth_lim": "0.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_auth_res": "200.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_auth_wgt": "100.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_data_lim": "0.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_data_res": "500.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_data_wgt": "500.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_metadata_lim": "0.000000", 2024-05-08T17:16:40.834 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_metadata_res": "500.000000", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dmclock_metadata_wgt": "500.000000", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dns_name": "", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dns_s3website_name": "", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_dynamic_resharding": "true", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_apis": "s3, s3website, swift, swift_auth, admin, sts, iam, notifications", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_gc_threads": "true", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_lc_threads": "true", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_ops_log": "false", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_quota_threads": "true", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_static_website": "false", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enable_usage_log": "false", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_enforce_swift_acls": "true", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_exit_timeout_secs": "120", 2024-05-08T17:16:40.835 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_expose_bucket": "false", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_extended_http_attrs": "", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_filter": "none", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_frontend_defaults": "beast ssl_certificate=config://rgw/cert/$realm/$zone.crt ssl_private_key=config://rgw/cert/$realm/$zone.key", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_frontends": "beast port=7480", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_concurrent_io": "10", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_deferred": "50", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_deferred_entries_size": "3072", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_objs": "32", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_queue_size": "134213632", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_max_trim_chunk": "16", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_obj_min_wait": "7200", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_processor_max_time": "3600", 2024-05-08T17:16:40.836 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_gc_processor_period": "3600", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_get_obj_max_req_size": "4194304", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_get_obj_window_size": "16777216", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_healthcheck_disabling_path": "", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ignore_get_invalid_range": "false", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_init_timeout": "300", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_inject_notify_timeout_probability": "0.000000", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_json_config": "/var/lib/ceph/radosgw/config.json", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_kafka_connection_idle": "300", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_kafka_message_timeout": "5000", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_kafka_sleep_timeout": "10", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_accepted_admin_roles": "", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_accepted_reader_roles": "", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_accepted_roles": "Member, admin", 2024-05-08T17:16:40.837 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_domain": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_password": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_password_path": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_project": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_tenant": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_token": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_token_path": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_admin_user": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_api_version": "2", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_barbican_domain": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_barbican_password": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_barbican_project": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_barbican_tenant": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_barbican_user": "", 2024-05-08T17:16:40.838 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_expired_token_cache_expiration": "3600", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_implicit_tenants": "false", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_service_token_accepted_roles": "admin", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_service_token_enabled": "false", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_token_cache_size": "10000", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_url": "", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_keystone_verify_ssl": "true", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_debug_interval": "-1", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_lock_max_time": "90", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_max_objs": "32", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_max_rules": "1000", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_max_worker": "3", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_max_wp_worker": "3", 2024-05-08T17:16:40.839 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lc_thread_delay": "0", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_binddn": "uid=admin,cn=users,dc=example,dc=com", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_dnattr": "uid", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_searchdn": "cn=users,cn=accounts,dc=example,dc=com", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_searchfilter": "", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_secret": "/etc/openldap/secret", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ldap_uri": "", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lifecycle_work_time": "00:00-06:00", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_list_bucket_min_readahead": "1000", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_list_buckets_max_chunk": "1000", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_log_http_headers": "", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_log_nonexistent_bucket": "false", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_log_object_name": "%Y-%m-%d-%H-%i-%n", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_log_object_name_utc": "false", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_lua_max_memory_per_state": "128000", 2024-05-08T17:16:40.840 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_luarocks_location": "/tmp/rgw_luarocks/mon.a", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_attr_name_len": "0", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_attr_size": "0", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_attrs_num_in_req": "0", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_chunk_size": "4194304", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_concurrent_requests": "1024", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_copy_obj_concurrent_io": "10", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_dynamic_shards": "1999", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_listing_results": "1000", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_notify_retries": "10", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_objs_per_shard": "100000", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_put_param_size": "1048576", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_put_size": "5368709120", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_max_slo_entries": "1000", 2024-05-08T17:16:40.841 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_md_log_max_shards": "64", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_md_notify_interval_msec": "200", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_meta_sync_poll_interval": "20", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_meta_sync_spawn_window": "20", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_mime_types_file": "/etc/mime.types", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_mp_lock_max_time": "600", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_multi_obj_del_max_aio": "16", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_multipart_min_part_size": "5242880", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_multipart_part_upload_limit": "10000", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_fhcache_partitions": "3", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_fhcache_size": "2017", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_frontends": "rgw-nfs", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_lru_lane_hiwat": "911", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_lru_lanes": "5", 2024-05-08T17:16:40.842 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_max_gc": "300", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_namespace_expire_secs": "300", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_run_gc_threads": "false", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_run_lc_threads": "false", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_run_quota_threads": "false", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_run_sync_thread": "false", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_s3_fast_attrs": "false", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_nfs_write_completion_interval_s": "10", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_num_async_rados_threads": "32", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_num_control_oids": "8", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_numa_node": "-1", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_obj_stripe_size": "4194304", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_obj_tombstone_cache_size": "1000", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_objexp_chunk_size": "100", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_objexp_gc_interval": "600", 2024-05-08T17:16:40.843 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_objexp_hints_num_shards": "127", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_olh_pending_timeout_sec": "3600", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_op_thread_suicide_timeout": "0", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_op_thread_timeout": "600", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_op_tracing": "false", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_opa_token": "", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_opa_url": "", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_opa_verify_ssl": "true", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ops_log_data_backlog": "5242880", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ops_log_file_path": "/var/log/ceph/ops-log-ceph-mon.a.log", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ops_log_rados": "false", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_ops_log_socket_path": "", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_override_bucket_index_max_shards": "0", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_pending_bucket_index_op_expiration": "120", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_period_latest_epoch_info_oid": ".latest_epoch", 2024-05-08T17:16:40.844 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_period_push_interval": "2.000000", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_period_push_interval_max": "30.000000", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_period_root_pool": ".rgw.root", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_policy_reject_invalid_principals": "true", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_base_path": "/tmp/rgw_posix_driver", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_cache_lanes": "3", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_cache_lmdb_count": "3", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_cache_max_buckets": "100", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_cache_partitions": "3", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_posix_database_root": "/var/lib/ceph/radosgw", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_print_continue": "true", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_print_prohibited_content_length": "false", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_put_obj_max_window_size": "67108864", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_put_obj_min_window_size": "16777216", 2024-05-08T17:16:40.845 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_rados_pool_autoscale_bias": "4.000000", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_rados_pool_recovery_priority": "5", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_rados_tracing": "false", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_realm": "", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_realm_id": "", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_realm_root_pool": ".rgw.root", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_region": "", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_region_root_pool": ".rgw.root", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_relaxed_region_enforcement": "false", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_relaxed_s3_bucket_names": "false", 2024-05-08T17:16:40.846 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_remote_addr_param": "REMOTE_ADDR", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_request_uri": "", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_reshard_batch_size": "64", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_reshard_bucket_lock_duration": "360", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_reshard_max_aio": "128", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_reshard_num_logs": "16", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_reshard_thread_interval": "600", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_resolve_cname": "false", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_rest_getusage_op_compat": "false", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_run_sync_thread": "true", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_disable_signature_url": "false", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_order": "sts, external, local", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_use_keystone": "false", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_use_ldap": "false", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_use_rados": "true", 2024-05-08T17:16:40.847 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_auth_use_sts": "false", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_client_max_sig_ver": "-1", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_s3_success_create_obj_status": "0", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_safe_max_objects_per_shard": "102400", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_scheduler_type": "throttler", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_script_uri": "", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_service_provider_name": "", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_shard_warning_threshold": "90.000000", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_client_id": "", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_client_secret": "", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_entry": "sts", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_key": "sts", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_max_session_duration": "43200", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_min_session_duration": "900", 2024-05-08T17:16:40.848 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sts_token_introspection_url": "", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_account_in_url": "false", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_auth_entry": "auth", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_auth_url": "", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_custom_header": "", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_enforce_content_length": "false", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_need_stats": "true", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_tenant_name": "", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_token_expiration": "86400", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_url": "", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_url_prefix": "swift", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_swift_versioning_enabled": "false", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_data_full_inject_err_probability": "0.000000", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_data_inject_err_probability": "0.000000", 2024-05-08T17:16:40.849 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_lease_period": "120", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_log_trim_concurrent_buckets": "4", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_log_trim_interval": "1200", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_log_trim_max_buckets": "16", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_log_trim_min_cold_buckets": "4", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_meta_inject_err_probability": "0.000000", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_obj_etag_verify": "false", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_trace_history_size": "4096", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_trace_per_node_log_size": "32", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_sync_trace_servicemap_update_interval": "10", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_thread_pool_size": "512", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_topic_persistency_max_retries": "0", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_topic_persistency_sleep_duration": "0", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_topic_persistency_time_to_live": "0", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_topic_require_publish_policy": "false", 2024-05-08T17:16:40.850 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_comment": "", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_createby": "", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_encoding": "", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_flag": "false", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_max_size": "5368709120", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_origin": "", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_sha_unit": "524288", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_torrent_tracker": "", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_trust_forwarded_https": "false", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_usage_log_flush_threshold": "1024", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_usage_log_tick_interval": "30", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_usage_max_shards": "32", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_usage_max_user_shards": "1", 2024-05-08T17:16:40.851 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_use_opa_authz": "false", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_counters_cache": "false", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_counters_cache_size": "10000", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_default_quota_max_objects": "-1", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_default_quota_max_size": "-1", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_max_buckets": "1000", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_policies_max_num": "100", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_quota_bucket_sync_interval": "180", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_quota_sync_idle_users": "false", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_quota_sync_interval": "86400", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_quota_sync_wait_time": "86400", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_user_unique_email": "true", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_verify_ssl": "true", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_website_routing_rules_max_num": "50", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zone": "", 2024-05-08T17:16:40.852 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zone_id": "", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zone_root_pool": ".rgw.root", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zonegroup": "", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zonegroup_id": "", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgw_zonegroup_root_pool": ".rgw.root", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgwlc_auto_session_clear": "true", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rgwlc_skip_bucket_step": "false", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_block_size": "4096", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_bloom_bits_per_key": "20", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_index_and_filter_blocks": "true", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_index_and_filter_blocks_with_high_priority": "false", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_row_ratio": "0.000000", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_shard_bits": "4", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_size": "536870912", 2024-05-08T17:16:40.853 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cache_type": "binned_lru", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cf_compact_on_deletion": "true", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cf_compact_on_deletion_sliding_window": "32768", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_cf_compact_on_deletion_trigger": "16384", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_collect_compaction_stats": "false", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_collect_extended_stats": "false", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_collect_memory_stats": "false", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_delete_range_threshold": "1048576", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_index_type": "binary_search", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_log_to_ceph_log": "true", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_metadata_block_size": "4096", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_partition_filters": "false", 2024-05-08T17:16:40.854 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_perf": "false", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "rocksdb_pin_l0_filter_and_index_blocks_in_cache": "false", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "rotating_keys_bootstrap_timeout": "30", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "rotating_keys_renewal_timeout": "10", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "run_dir": "/var/run/ceph", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_block_create": "true", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_cache_lru_size": "67108864", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_cbjournal_size": "5368709120", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_data_delta_based_overwrite": "0", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_default_max_object_size": "16777216", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_default_object_metadata_reservation": "16777216", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_device_size": "53687091200", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_journal_batch_capacity": "16", 2024-05-08T17:16:40.855 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_journal_batch_flush_size": "16777216", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_journal_batch_preferred_fullness": "0.950000", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_journal_iodepth_limit": "5", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_main_device_type": "SSD", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_max_concurrent_transactions": "8", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_multiple_tiers_default_evict_ratio": "0.600000", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_multiple_tiers_fast_evict_ratio": "0.700000", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_multiple_tiers_stop_evict_ratio": "0.500000", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_obj_data_write_amplification": "1.250000", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "seastore_segment_size": "67108864", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "set_keepcaps": "false", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "setgroup": "", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "setuser": "", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "setuser_match_path": "", 2024-05-08T17:16:40.856 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_misplaced_ratio": "0.050000", 2024-05-08T17:16:40.857 INFO:teuthology.orchestra.run.smithi154.stdout: "thp": "false", 2024-05-08T17:16:40.857 INFO:teuthology.orchestra.run.smithi154.stdout: "threadpool_default_timeout": "60", 2024-05-08T17:16:40.857 INFO:teuthology.orchestra.run.smithi154.stdout: "threadpool_empty_queue_max_wait": "2", 2024-05-08T17:16:40.857 INFO:teuthology.orchestra.run.smithi154.stdout: "throttler_perf_counter": "true" 2024-05-08T17:16:40.857 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:40.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=0 2024-05-08T17:16:40.976 INFO:teuthology.orchestra.run.smithi154.stdout:mon.a: {} 2024-05-08T17:16:40.976 INFO:teuthology.orchestra.run.smithi154.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:16:40.986 INFO:teuthology.orchestra.run.smithi154.stdout:mon.b: {} 2024-05-08T17:16:40.988 INFO:teuthology.orchestra.run.smithi154.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:16:40.995 INFO:teuthology.orchestra.run.smithi154.stdout:mon.c: {} 2024-05-08T17:16:40.995 INFO:teuthology.orchestra.run.smithi154.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:16:41.006 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-05-08T17:16:41.024 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T17:16:41.024 DEBUG:teuthology.orchestra.run.smithi154:> type ceph-objectstore-tool 2024-05-08T17:16:41.039 INFO:teuthology.orchestra.run.smithi154.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T17:16:41.039 INFO:teuthology.run_tasks:Running task mon_recovery... 2024-05-08T17:16:41.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:16:41.051 INFO:tasks.thrashosds.thrasher:starting do_thrash 2024-05-08T17:16:41.051 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:16:41.051 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:16:41.051 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T17:16:41.051 INFO:tasks.thrashosds.thrasher:Setting osd 0 primary_affinity to 1.000000 2024-05-08T17:16:41.052 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-08T17:16:41.052 INFO:tasks.thrashosds.thrasher:starting do_sighup with a delay of 0.1 2024-05-08T17:16:41.052 INFO:tasks.thrashosds.thrasher:starting do_optrack_toggle with a delay of 2.0 2024-05-08T17:16:41.052 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:16:41.053 INFO:tasks.thrashosds.thrasher:starting do_dump_ops 2024-05-08T17:16:41.053 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:41.053 INFO:tasks.thrashosds.thrasher:starting do_noscrub_toggle with a delay of 2.0 2024-05-08T17:16:41.054 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:16:41.081 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:41.084+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:41.154 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:41.158+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:41.219 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:16:41.220 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:16:41.243 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.243 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:41.257 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:16:41.257 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:16:41.257 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:41.260+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:41.292 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:16:41.293 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:16:41.327 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:16:41.328 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:16:41.356 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:41.359+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:41.362 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:16:41.362 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:16:41.365 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.366 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:41.399 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:16:41.399 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:16:41.400 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:16:41.400 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":17,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:16:35.854143+0000","last_up_change":"2024-05-08T17:16:32.786322+0000","last_in_change":"2024-05-08T17:16:24.632481+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":3448847701}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":3448847701}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":3448847701}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":3448847701}]},"public_addr":"172.21.15.183:6801/3448847701","cluster_addr":"172.21.15.183:6803/3448847701","heartbeat_back_addr":"172.21.15.183:6807/3448847701","heartbeat_front_addr":"172.21.15.183:6805/3448847701","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":3679914992}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":3679914992}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":3679914992}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":3679914992}]},"public_addr":"172.21.15.183:6800/3679914992","cluster_addr":"172.21.15.183:6802/3679914992","heartbeat_back_addr":"172.21.15.183:6806/3679914992","heartbeat_front_addr":"172.21.15.183:6804/3679914992","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2695818974}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2695818974}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2695818974}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2695818974}]},"public_addr":"172.21.15.183:6808/2695818974","cluster_addr":"172.21.15.183:6809/2695818974","heartbeat_back_addr":"172.21.15.183:6811/2695818974","heartbeat_front_addr":"172.21.15.183:6810/2695818974","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:16:41.418 INFO:tasks.mon_recovery.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T17:16:32.734319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T17:16:33.268021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.9900000095367432, 'score_stable': 1.9900000095367432, 'optimal_score': 0.8799999952316284, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T17:16:41.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T17:16:41.436 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:16:41.436 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:16:41.456 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:41.460+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:41.473 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:16:41.474 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:16:41.510 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.510 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:41.557 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:41.561+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:41.658 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:41.662+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:41.667 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.667 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.678 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:41.736 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 1 2024-05-08T17:16:41.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2024-05-08T17:16:41.792 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:41.795+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:41.826 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.826 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:41.877 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:41.880+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:41.893 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-08T17:16:41.896 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:16:41.961 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:41.964+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:41.989 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:41.989 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:41.998 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:42.061 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:42.064+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:42.097 INFO:teuthology.orchestra.run.smithi154.stdout:pg_num: 8 2024-05-08T17:16:42.111 INFO:tasks.mon_recovery:mon ids = ['a', 'c', 'b'] 2024-05-08T17:16:42.111 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:16:42.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:42.148 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.148 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.157 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:42.162 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:42.165+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:42.263 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:42.266+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:42.305 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.305 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:42.364 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:42.367+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:42.420 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.420 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.429 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:42.472 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:42.476+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:42.474 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":20,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:42.488 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:16:42.488 INFO:tasks.mon_recovery:verifying all monitors are in the quorum 2024-05-08T17:16:42.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.a mon_status 2024-05-08T17:16:42.537 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.537 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:42.567 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:42.570+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:42.646 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "a", 2024-05-08T17:16:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 0, 2024-05-08T17:16:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "leader", 2024-05-08T17:16:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "election_epoch": 8, 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 26532, 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum": [ 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: 0, 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: 1, 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: 2 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_age": 20, 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "required_con": "2450521708007866372", 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "required_mon": [ 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_con": "4540701547738038271", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_mon": [ 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "outside_quorum": [], 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "extra_probe_peers": [], 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "sync_provider": [], 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "monmap": { 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 1, 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:16:42.649 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release": 19, 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release_name": "squid", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "election_strategy": 3, 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "disallowed_leaders: ": "", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false, 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "tiebreaker_mon": "", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_ranks: ": "", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "persistent": [ 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.650 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "optional": [] 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "mons": [ 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 0, 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "a", 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.651 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3300", 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789", 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 1, 2024-05-08T17:16:42.653 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "b", 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:3300", 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789", 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 2, 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "c", 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3301", 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790", 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "feature_map": { 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": [ 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": [ 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.657 INFO:teuthology.orchestra.run.smithi154.stdout: "client": [ 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 5 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false 2024-05-08T17:16:42.658 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:42.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.c mon_status 2024-05-08T17:16:42.696 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.696 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.697 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:42.700+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:42.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:42.777 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "c", 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 2, 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "peon", 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "election_epoch": 8, 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 26663, 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum": [ 2024-05-08T17:16:42.778 INFO:teuthology.orchestra.run.smithi154.stdout: 0, 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: 1, 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: 2 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_age": 20, 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "required_con": "2450521708007866372", 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "required_mon": [ 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.779 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.780 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_con": "4540701547738038271", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_mon": [ 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.781 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "outside_quorum": [], 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "extra_probe_peers": [], 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "sync_provider": [], 2024-05-08T17:16:42.782 INFO:teuthology.orchestra.run.smithi154.stdout: "monmap": { 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 1, 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release": 19, 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release_name": "squid", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "election_strategy": 3, 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "disallowed_leaders: ": "", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false, 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "tiebreaker_mon": "", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_ranks: ": "", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "persistent": [ 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.783 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "optional": [] 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: "mons": [ 2024-05-08T17:16:42.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 0, 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "a", 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3300", 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789", 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.785 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 1, 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "b", 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.786 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:3300", 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789", 2024-05-08T17:16:42.787 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 2, 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "c", 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.788 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3301", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.789 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "feature_map": { 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": [ 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": [ 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.790 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 3 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "client": [ 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 2 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": [ 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.791 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false 2024-05-08T17:16:42.792 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:42.792 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:42.781+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:42.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mon.b mon_status 2024-05-08T17:16:42.856 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.857 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:42.870 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:42.873+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:42.950 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "b", 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 1, 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "state": "peon", 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "election_epoch": 8, 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "uptime": 26797, 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum": [ 2024-05-08T17:16:42.951 INFO:teuthology.orchestra.run.smithi154.stdout: 0, 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: 1, 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: 2 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_age": 20, 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "required_con": "2450521708007866372", 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "required_mon": [ 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.952 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_con": "4540701547738038271", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "quorum_mon": [ 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.954 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "outside_quorum": [], 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "extra_probe_peers": [], 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "sync_provider": [], 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "monmap": { 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 1, 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:15:43.320397Z", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release": 19, 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "min_mon_release_name": "squid", 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "election_strategy": 3, 2024-05-08T17:16:42.955 INFO:teuthology.orchestra.run.smithi154.stdout: "disallowed_leaders: ": "", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false, 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "tiebreaker_mon": "", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_ranks: ": "", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "features": { 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "persistent": [ 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "kraken", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "luminous", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "mimic", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "osdmap-prune", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "nautilus", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "octopus", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "pacific", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "elector-pinging", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "quincy", 2024-05-08T17:16:42.956 INFO:teuthology.orchestra.run.smithi154.stdout: "reef", 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "squid" 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "optional": [] 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "mons": [ 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 0, 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "a", 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3300", 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789", 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.958 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6789/0", 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.959 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 1, 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "b", 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:3300", 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789", 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6789/0", 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "rank": 2, 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "name": "c", 2024-05-08T17:16:42.961 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:3301", 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v1", 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790", 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 0 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.962 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6790/0", 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "priority": 0, 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 0, 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_location": "{}" 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "feature_map": { 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "mon": [ 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.963 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": [ 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 4 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "client": [ 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:16:42.964 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": [ 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: "features": "0x3f03cffffffdffff", 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: "release": "luminous", 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: "num": 1 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": false 2024-05-08T17:16:42.965 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:42.966 INFO:tasks.mon_recovery:restarting each monitor in turn 2024-05-08T17:16:42.967 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T17:16:42.967 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:16:42.969 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:42.972+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:42.979 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T17:16:42.979 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:16:42.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:42.986 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:42.986 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:42.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:43.070 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:43.073+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:43.101 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:43.101 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:43.111 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:43.195 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:43.198+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:43.257 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:43.257 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:43.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:43.307 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:43.310+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:43.372 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:43.376+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:43.375 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":21,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:43.410 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:43.410 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:43.419 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:16:43.473 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:43.477+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:43.490 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:16:43.524 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:43.524 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:43.533 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:16:43.577 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:43.580+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:43.613 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:16:43.613 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:16:43.631 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:16:43.631 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:16:43.638 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:43.639 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:43.647 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:16:43.647 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:16:43.648 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:16:43.663 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:16:43.663 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:16:43.680 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:16:43.680 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:16:43.689 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:43.692+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:43.697 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:16:43.697 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:16:43.714 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:16:43.714 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:16:43.731 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:16:43.731 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:16:43.777 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:43.780+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:43.794 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:43.794 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:43.794 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:16:43.794 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:43.802 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:16:43.879 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:43.882+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:43.908 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:43.908 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:43.908 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:16:43.908 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:16:43.909 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:43.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:16:43.917 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:16:43.979 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:43.982+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:44.021 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:16:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:16:44.021 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:16:44.021 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:44.030 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:16:44.080 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:44.083+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:44.134 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:44.134 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:44.134 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:16:44.135 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:44.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:16:44.181 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:44.184+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:44.247 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:44.248 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:44.248 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:16:44.248 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:16:44.248 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:44.256 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:16:44.301 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:44.304+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.563027134, 2024-05-08T17:16:44.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:16:44.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:16:44.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:16:44.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:16:44.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:16:44.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.530063543000001, 2024-05-08T17:16:44.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:16:44.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:16:44.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:16:44.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.512299128, 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:16:44.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.512206098, 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:16:44.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:16:44.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:16:44.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.484108073, 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.483997995999999, 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:16:44.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:16:44.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.460137467999999, 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.318777860000001, 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:16:44.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.318660664999999, 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:16:44.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 10.318635713999999, 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.416 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:16:44.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:16:44.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9207648559999999, 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9207492760000004, 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:16:44.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:16:44.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9207299560000006, 2024-05-08T17:16:44.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:16:44.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:16:44.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9206534529999999, 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:16:44.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.444 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9206318580000001, 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.445 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:16:44.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.450 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9206054639999994, 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.9205312380000006, 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.456 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4347377409999993, 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4347244539999995, 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.467 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:16:44.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:16:44.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:16:44.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 9.4346553859999993, 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:16:44.473 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:16:44.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:16:44.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:16:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:16:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:16:44.479 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:44.480 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:44.483+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:44.481 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:44.582 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:44.584+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:44.586 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:44.586 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:44.586 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:44.586 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:44.595 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:44.683 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:44.686+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:44.699 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:44.699 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:44.699 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:44.699 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:44.700 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:44.708 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:44.784 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:44.787+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.053972366, 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:16:44.857 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:16:44.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:16:44.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.031999214000001, 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.862 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:44.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:44.868 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.031542858, 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:16:44.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:16:44.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.012501102, 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:16:44.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.004212043000001, 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:16:44.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.003724039, 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:44.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.003249606000001, 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:16:44.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:16:44.892 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 11.002856231000001, 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.996669794000001, 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.897 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:44.901 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.991866343, 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.905 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.989290884999999, 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:16:44.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.910 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.986505189000001, 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.981502566, 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.810326505000001, 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.919 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.399742535, 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:16:44.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:16:44.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:16:44.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9269454239999995, 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9263598579999996, 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:16:44.932 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:16:44.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:16:44.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:16:44.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9259204509999996, 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9252724659999991, 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.941 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:16:44.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:16:44.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.945 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.9198107390000008, 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:44.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:44.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:44.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:16:44.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:16:44.950 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.950 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.950 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.950 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:44.950 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:44.952 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:44.953 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:44.953 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:44.956+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:45.053 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:45.056+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:45.100 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.100 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:45.101 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:45.101 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.109 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:45.155 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:45.157+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:45.215 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.216 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:45.216 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:45.216 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:45.216 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.225 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:45.255 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:45.258+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:45.356 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:45.359+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:45.371 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.371 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:45.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:45.371 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:16:45.371 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:45.457 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:45.460+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:45.485 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:45.486 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.494 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:45.578 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:45.581+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:45.642 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:45.642 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:45.642 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:45.692 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:45.694+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:45.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:16:45.759 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:45.762+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:45.799 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:16:45.799 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.809 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:45.861 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:45.864+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:45.870 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:16:45.871 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:16:45.888 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:16:45.888 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:16:45.905 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:16:45.905 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:16:45.917 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:45.918 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:45.918 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:45.918 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:45.922 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:16:45.922 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:16:45.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:45.941 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:16:45.941 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:16:45.959 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:16:45.959 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:16:45.961 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:45.964+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:45.977 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:16:45.977 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:16:45.995 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:16:45.995 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:16:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.035 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:46.063 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:46.066+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:46.164 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:46.167+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:46.190 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.191 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:46.264 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:46.267+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:46.317 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.317 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.327 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:46.366 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:46.368+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:46.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:46.467 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:46.470+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:46.472 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.472 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.481 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:46.568 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:46.571+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:46.627 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.627 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.636 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:46.678 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:46.680+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:46.741 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":24,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:46.770 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:46.773+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:46.783 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.784 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:16:46.871 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:46.874+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:46.896 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:46.896 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:46.905 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:16:46.915 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:16:46.916 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:16:46.916 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T17:16:46.916 INFO:tasks.thrashosds.thrasher:Setting osd 5 primary_affinity to 0.000000 2024-05-08T17:16:46.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 0 2024-05-08T17:16:46.972 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:46.975+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:47.050 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.050 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:16:47.102 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:47.105+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:47.164 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.164 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:16:47.214 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:47.216+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:47.274 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:47.277+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:47.316 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.316 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:16:47.375 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:47.378+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:47.431 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.431 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.440 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:16:47.476 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:47.479+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:47.577 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:47.580+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:47.586 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.586 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.595 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:16:47.679 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:47.682+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:47.741 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.741 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:16:47.791 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:47.794+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:47.879 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:47.882+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:47.894 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:47.894 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:47.904 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:47.980 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:47.983+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:48.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:16:48.013 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.013 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.022 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:48.080 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:48.083+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:48.170 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.170 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:48.182 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:48.184+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:48.282 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:48.285+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:48.285 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.285 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.293 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:48.383 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:48.386+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:48.398 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.399 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.407 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:48.484 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:48.486+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:48.512 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.512 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:48.605 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:48.607+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:48.666 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.666 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.675 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:48.686 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:48.689+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:48.786 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:48.789+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:48.823 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.824 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:48.916 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:48.919+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:48.979 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:48.979 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:48.988 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:48.988 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:48.991+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:49.088 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:49.091+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:49.133 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.134 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:49.189 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:49.191+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:49.248 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.248 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.258 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:49.290 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:49.293+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:49.391 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:49.394+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:49.403 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.403 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.412 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:49.492 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:49.495+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:49.517 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.517 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:49.593 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:49.595+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:49.643 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.643 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.653 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:49.693 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:49.696+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:49.755 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:49.794 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:49.797+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:49.799 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.799 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.808 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:49.895 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:49.898+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:49.954 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:49.954 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:49.964 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:49.997 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:49.999+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:50.069 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.070 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:16:50.113 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":8,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":27,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:50.123 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:50.126+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:50.184 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.184 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:16:50.234 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:50.237+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:50.299 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:50.302+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:50.338 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.339 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.347 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:16:50.400 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:50.403+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:50.452 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.452 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.460 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:16:50.501 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:50.504+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:50.602 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:50.605+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:50.604 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.604 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.613 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:16:50.703 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:50.705+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:50.760 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.760 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.769 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:16:50.804 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:50.807+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:50.905 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:50.907+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:50.916 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:50.916 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:50.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:16:51.006 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:51.009+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:51.069 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.070 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:16:51.107 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:51.109+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:51.183 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.218 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:51.220 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:51.222+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:51.321 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:51.323+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:51.337 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.338 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:51.421 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:51.423+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:51.454 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.454 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:51.547 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:51.550+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:51.609 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.609 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:51.624 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:51.627+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:51.725 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:51.727+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:51.765 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.765 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:51.825 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:51.827+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:51.879 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:51.879 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:51.888 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:51.929 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:51.932+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:52.027 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:52.030+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:52.034 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.035 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:52.130 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:52.133+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:52.150 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.150 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:52.232 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:52.235+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:52.305 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.305 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:52.357 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:52.360+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:52.420 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.420 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.429 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:52.471 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:52.473+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:52.539 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:52.541+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:52.575 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.575 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.583 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:52.667 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:52.670+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:52.730 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.730 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:52.744 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:52.742+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:52.840 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:52.843+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:52.890 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:52.890 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:52.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:52.942 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:52.945+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:53.016 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.016 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:53.043 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:53.046+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:53.126 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:53.131 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.131 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.141 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:53.171 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:53.173+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:53.244 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:53.247+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:53.246 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.247 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.256 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:53.345 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:53.348+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:53.402 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.402 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:16:53.446 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:53.449+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:53.547 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:53.550+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:53.555 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.555 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:16:53.648 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:53.651+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:53.669 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.669 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:16:53.762 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:53.765+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:53.823 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.823 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.831 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:16:53.873 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:53.876+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:53.951 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:53.954+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:53.977 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:53.977 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:53.986 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:16:54.052 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:54.055+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:54.091 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.091 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:16:54.185 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:54.187+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:54.245 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.245 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:16:54.254 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:54.256+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:54.354 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:54.357+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:54.399 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.400 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.409 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:16:54.457 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:54.459+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:54.514 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.514 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.523 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:54.557 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:54.560+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:54.641 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.642 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.651 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:54.658 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:54.661+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:54.756 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.756 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.759 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:54.762+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:54.765 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:54.861 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:54.864+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:54.869 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.870 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:54.962 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:54.965+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:54.983 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:54.983 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:54.992 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:55.064 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:55.066+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:55.097 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.097 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:55.164 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:55.167+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:55.212 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.212 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:55.265 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:55.268+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:55.326 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.327 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.335 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:55.366 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:55.369+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:55.466 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:55.469+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:55.480 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.480 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:55.567 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:55.570+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:55.595 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.596 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:55.668 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:55.671+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:55.751 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.751 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.760 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:55.768 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:55.771+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:55.865 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.865 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.869 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:55.872+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:55.874 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:55.970 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:55.973+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:55.978 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:55.978 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:55.987 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:56.071 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:56.074+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:56.104 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.104 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.114 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:56.198 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:56.201+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:56.261 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.261 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.269 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:56.311 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:56.314+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:56.374 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:56.377+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:56.415 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.415 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.424 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:56.476 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:56.479+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:56.569 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.569 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.577 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:56.580+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:56.578 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:16:56.677 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:56.680+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:56.682 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.682 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.691 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:16:56.778 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:56.781+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:56.836 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.836 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:56.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:16:56.886 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:56.889+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:56.980 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:56.983+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:56.991 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:56.991 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.000 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:16:57.081 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:57.084+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:57.104 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.104 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.112 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:16:57.196 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:57.199+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:57.258 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.258 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.267 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:16:57.308 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:57.311+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:57.384 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:57.387+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:57.414 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.414 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.423 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:16:57.484 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:57.487+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:57.528 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.528 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:16:57.586 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:57.589+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:57.683 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.683 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.686 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:16:57.689+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:16:57.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:16:57.787 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:57.790+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:57.812 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:16:57.812 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:16:57.822 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:16:57.889 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:57.892+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:57.920 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:16:57.921 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:16:57.939 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:16:57.939 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:16:57.956 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:16:57.956 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:16:57.970 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:16:57.971 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.5 primary-affinity to 0 (802) 2024-05-08T17:16:57.976 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:16:57.976 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:16:57.978 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:57.978 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:57.978 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:57.979 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:57.979 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:57.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:16:57.990 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:57.992+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:58.010 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:16:58.010 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:16:58.031 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:16:58.031 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:16:58.049 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:16:58.050 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:16:58.067 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:16:58.067 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.121 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.318199521, 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.125 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.296226368999999, 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:16:58.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.295770012999998, 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.130 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:16:58.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.276728256999998, 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:16:58.134 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:16:58.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.138 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.268439197999999, 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:58.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:16:58.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:16:58.142 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.267951193999998, 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:58.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:16:58.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:16:58.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.267476761000001, 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:58.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.267083385999999, 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.151 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:16:58.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:16:58.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.260896948999999, 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:16:58.155 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:16:58.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.159 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.256093497999998, 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:16:58.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:16:58.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.253518039999999, 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:16:58.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.250732343999999, 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.168 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:58.172 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.245729721, 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.074553659999999, 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:16:58.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.663969689999998, 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.191172579, 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.185 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:16:58.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:16:58.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:16:58.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.190587012999998, 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:16:58.189 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:16:58.193 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.190147606, 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.197 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.189499621, 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 23.184037893999999, 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:16:58.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:16:58.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:16:58.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:16:58.206 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.206 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:58.123+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:58.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:16:58.208 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":10,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:58.213 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:16:58.214 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T17:16:58.214 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T17:16:58.214 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:58.215+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:58.256 INFO:tasks.ceph.mon.a:Started 2024-05-08T17:16:58.257 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:16:58.257 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:16:58.340 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:58.343+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:58.364 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.364 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:58.364 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:58.364 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.373 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:16:58.415 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:16:58.418+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:16:58.512 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:58.515+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:58.522 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:58.522 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:58.522 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.531 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:16:58.613 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:58.616+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:58.662 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":10,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:16:58.679 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.679 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:58.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:58.679 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:16:58.679 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.688 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:16:58.715 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:16:58.718+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:16:58.816 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:58.819+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:58.835 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.835 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:58.835 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:58.835 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:16:58.917 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:58.919+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:58.952 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:58.952 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:58.952 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:58.952 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:58.952 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:58.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:16:59.017 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:59.021+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:59.108 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:59.109 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:59.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:59.109 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:16:59.109 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:59.118 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:16:59.119 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:59.120+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:59.219 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:59.221+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:59.265 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:59.265 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:59.265 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:16:59.265 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:59.274 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:16:59.319 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:59.322+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:59.419 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:59.419 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:16:59.419 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:16:59.419 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:16:59.420 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:59.420 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:16:59.422+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:16:59.428 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:16:59.521 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:59.523+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:59.574 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:16:59.574 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:16:59.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:16:59.575 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:16:59.575 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:16:59.583 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:16:59.627 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:16:59.630+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:16:59.701 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:59.701 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:16:59.701 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:59.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:16:59.722 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:16:59.725+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:16:59.824 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:16:59.827+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:16:59.855 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:59.856 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:16:59.856 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:16:59.856 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:16:59.856 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:59.864 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:16:59.925 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:16:59.928+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:16:59.969 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:16:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:16:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:16:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:16:59.970 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:16:59.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:16:59.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:17:00.063 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:00.066+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:00.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:17:00.125 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:00.125 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:00.126 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:00.127 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:00.129+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:00.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:00.206 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:00.207 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:00.225 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:00.226 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:00.228 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:00.230+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:00.242 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:00.243 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:00.259 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:00.259 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:00.277 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:00.277 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:00.281 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:00.281 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:00.281 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:00.281 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:00.281 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:00.291 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:00.294 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:17:00.294 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:17:00.311 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:00.311 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:00.329 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:00.329 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:00.334 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:00.337+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:00.396 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:00.396 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:00.405 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:00.446 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:00.449+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:00.530 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:00.533+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:00.549 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:00.549 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:00.559 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:00.631 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:00.634+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:00.664 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:00.664 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:00.672 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:00.732 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:00.735+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:00.777 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:00.778 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:00.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:00.870 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:00.873+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:00.933 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:00.933 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:00.934 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:00.937+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:00.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:01.035 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:01.038+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:01.090 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.090 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.097 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:01.103 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:17:01.159 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:01.162+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:01.203 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.203 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:01.257 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:01.260+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:01.320 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.321 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:01.371 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:01.374+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:01.438 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:01.441+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:01.475 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.475 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.484 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:01.539 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:01.542+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:01.632 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.632 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.640 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:01.643+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:01.641 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:01.677 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:01.741 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:01.744+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:01.750 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.750 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:01.842 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:01.844+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:01.872 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:01.872 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:01.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:01.942 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:01.946+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:02.030 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:02.030 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:02.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:02.043 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:02.046+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:02.058 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:02.072 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:17:02.073 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T17:17:02.073 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:17:02.126 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T17:17:02.127 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:17:02.127 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:02.145 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:02.148+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:02.152 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:02.152 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:02.163 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:02.247 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:02.250+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:02.310 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:02.310 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:02.319 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:02.341 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:17:02.361 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:02.363+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:02.448 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:02.450+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:02.459 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:17:02.461+0000 7fcdca7fc640 0 --2- 172.21.15.154:0/1631619358 >> [v2:172.21.15.154:3300/0,v1:172.21.15.154:6789/0] conn(0x7fcdcc0b86a0 0x7fcdcc145f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T17:17:02.470 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:02.470 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:02.483 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:02.487 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:02.488 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:02.506 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:02.506 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:02.524 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:02.524 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:02.542 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:02.543 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:02.549 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:02.551+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:02.561 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:02.561 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:02.578 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:17:02.579 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:17:02.596 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:02.596 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:02.597 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:02.597 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:02.597 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:02.597 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:02.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:02.614 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:02.614 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:02.650 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:02.653+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:02.721 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:02.722 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:02.722 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:02.722 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:02.722 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:02.731 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:02.732 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:02.787 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:02.789+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:02.837 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:02.837 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:02.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:02.837 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:02.846 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:02.891 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:02.894+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:02.952 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:02.955+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:02.964 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:02.964 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:02.964 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:02.973 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:02.989 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:17:02.990 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:17:02.990 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-05-08T17:17:02.990 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:17:02.990 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:17:02.990 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-05-08T17:17:03.052 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:03.056+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:03.118 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:17:03.123 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.123 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T17:17:03.123 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.123 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.123 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:03.124 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:03.124 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:03.124 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.134 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:03.178 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:03.181+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:03.239 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.239 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:03.240 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:03.240 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:03.255 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:03.258+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:03.356 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:03.359+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:03.394 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:03.394 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:03.394 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.403 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:03.457 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:03.461+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:03.506 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:03.506 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:03.507 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:03.507 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.515 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:03.559 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:03.562+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:03.619 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.619 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:03.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:03.619 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:03.619 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.627 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:03.668 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:03.671+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:03.761 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:03.764+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:03.774 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.774 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:03.774 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:03.774 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:03.868 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:03.871+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:03.929 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:03.929 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:03.929 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:03.929 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:03.929 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:03.937 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:03.963 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:03.966+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:04.064 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:04.068+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:04.082 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:04.082 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:04.082 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:04.091 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:04.165 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:04.168+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:04.196 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:04.196 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:04.196 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:04.197 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:04.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:04.266 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:04.269+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:04.311 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:04.311 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:04.311 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:04.319 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:04.367 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:04.370+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.467 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.666493956, 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:04.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:04.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:04.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:04.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.633530364999999, 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.474 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:04.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:04.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.61576595, 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.480 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:04.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:04.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:04.485 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.615672920000002, 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:04.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.491 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.587574894999999, 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:04.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:04.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:04.497 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.587464818000001, 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:04.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.563604290000001, 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.503 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:04.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:04.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.422244681999999, 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:04.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:04.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.422127487000001, 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:04.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.422102536000001, 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:04.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:04.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.526 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024231678, 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:04.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:04.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:04.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024216098, 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.532 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:04.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:04.537 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024196778, 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.538 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:04.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:04.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.543 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024120275000001, 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.544 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:04.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:04.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024098680000002, 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:04.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.024072285999999, 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.555 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.02399806, 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:04.561 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:04.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:04.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.538204563000001, 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:04.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:04.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.538191275999999, 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:04.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:04.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:04.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:04.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 29.538122208000001, 2024-05-08T17:17:04.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:04.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:04.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:04.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:04.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:04.584 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:04.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:04.586 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:04.589+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:04.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:17:04.686 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:04.689+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:04.706 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:04.706 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:04.707 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:04.707 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:04.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:04.801 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:04.805+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:04.863 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:04.864 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:04.864 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:04.872 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:04.887 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:04.890+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:04.988 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:04.991+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:05.020 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.217218057, 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:05.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:05.024 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.195244904999999, 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.194788548999998, 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.029 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:05.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:05.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.175746792999998, 2024-05-08T17:17:05.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:05.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:05.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:05.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.167457733999999, 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:05.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:05.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.166969730000002, 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.045 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:05.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:05.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:05.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:05.049 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.166495297000001, 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:05.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:05.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:05.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.166101921999999, 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.159915484999999, 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.058 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:05.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:05.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:05.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.155112033999998, 2024-05-08T17:17:05.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.066 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.152536575999999, 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.149750879999999, 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.071 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:05.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:05.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:05.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.144748257, 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:05.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.973572195999999, 2024-05-08T17:17:05.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:05.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.084 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.562988226000002, 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:05.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:05.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.090191115, 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:05.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:05.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.093 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.089605549000002, 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:05.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:05.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.089166142, 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:05.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:05.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:05.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.088518156999999, 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.083056429999999, 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:05.106 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:05.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:05.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:05.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:05.110 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:05.112 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:05.115+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:05.212 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:05.216+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:05.259 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.259 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:05.259 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:05.259 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.267 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:05.351 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:05.355+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:05.413 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.413 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:05.413 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:05.413 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:05.413 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.414 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:05.418+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:05.422 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:05.516 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:05.519+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:05.569 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.569 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:05.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:05.569 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:05.569 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:05.616 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:05.620+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:05.684 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.684 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:05.684 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:05.684 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:05.737 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:05.741+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:05.746 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:05.803 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:05.803 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:05.803 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:05.803 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:05.857 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:05.861+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:05.919 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:05.922+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:05.922 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:05.922 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:05.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:05.922 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:05.922 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:05.932 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:06.019 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:06.023+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:06.080 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:06.080 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:06.080 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:06.080 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:06.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:06.114 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:06.127 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:06.130+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:06.218 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:06.218 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:06.219 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:06.219 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:06.219 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:06.222 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:06.225+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:06.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:06.323 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:06.326+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:06.332 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:06.332 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:06.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:06.332 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:06.332 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:06.341 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:06.424 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:06.427+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:06.459 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:06.460 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:06.460 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:06.460 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:06.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:06.525 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:06.528+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:06.615 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:06.616 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:06.616 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:06.624 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:06.626 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:06.628+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:06.726 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:06.730+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:06.730 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:06.730 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:06.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:06.730 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:06.730 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:06.739 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:06.827 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:06.831+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:06.844 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:06.845 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:06.845 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:06.845 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:06.854 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:06.928 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:06.931+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:06.958 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:06.958 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:06.958 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:06.958 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:06.958 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:06.967 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:07.029 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:07.032+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:07.112 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.112 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:07.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:07.112 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:07.112 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.121 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:07.130 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:07.134+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:07.225 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:07.225 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:07.226 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.231 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:07.234+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:07.234 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:07.332 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:07.335+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:07.380 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.380 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:07.380 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:07.380 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:07.380 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.388 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:07.472 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:07.475+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:07.533 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.534 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:07.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:07.534 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:07.534 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.534 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:07.537+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:07.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:07.635 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:07.638+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:07.688 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.688 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:07.688 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:07.688 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:07.740 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:07.743+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:07.802 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:07.802 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:07.802 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:07.811 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:07.837 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:07.841+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:07.938 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:07.941+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:07.959 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.158326909000003, 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:07.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:07.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.966 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.125363317999998, 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:07.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.971 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.107598903000003, 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.972 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:07.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:07.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:07.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:07.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.107505873000001, 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.979 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:07.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:07.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:07.986 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.986 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.079407848000002, 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.987 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:07.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:07.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.079297771, 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:07.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:07.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:07.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 34.055437243, 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:07.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:07.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:08.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:08.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.914077634999998, 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.004 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:08.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.913960439999997, 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.009 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:08.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:08.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:08.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:08.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.913935488999996, 2024-05-08T17:17:08.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:08.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:08.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.516064630999999, 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:08.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:08.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:08.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:08.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:08.025 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.516049051000003, 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.026 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:08.032 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.516029731000003, 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:08.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.037 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.515953228000001, 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:08.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:08.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.515931633000001, 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:08.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:08.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:08.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:08.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.515905238999999, 2024-05-08T17:17:08.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.515831013000003, 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.055 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:08.060 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.030037516, 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:08.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.030024228999999, 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.066 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.071 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 33.029955160999997, 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:08.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:08.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:08.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.077 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.077 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.077 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:08.077 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:08.077 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:08.078 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:08.081+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:08.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:08.134 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:17:08.134 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:17:08.179 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:08.183+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:08.196 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:08.196 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:08.196 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:08.280 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:08.283+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:08.352 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.352 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:08.352 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:08.352 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:08.352 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.361 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:08.381 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:08.384+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:08.468 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.468 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.665792230999998, 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:08.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:08.473 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.643819078999996, 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.477 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.643362723000003, 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:08.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:08.482 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.624320967000003, 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:08.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.616031907999997, 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:08.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.491 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.615543903999999, 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:08.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:08.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.615069470999998, 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:08.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.614676095999997, 2024-05-08T17:17:08.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:08.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.608489659, 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.504 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:08.508 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.603686207999999, 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:08.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:08.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.601110749999997, 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.598325054, 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:08.517 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:08.521 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.593322430999997, 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:08.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:08.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:08.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.42214637, 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:08.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.011562400000003, 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.530 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:08.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.538765288999997, 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:08.542 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.538179722999999, 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.547 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.537740315999997, 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:08.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:08.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.537092330999997, 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:08.552 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:08.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.556 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.531630604, 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:08.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:08.562 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:08.562 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:08.564 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:08.566+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:08.664 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:08.667+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:08.711 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:08.712 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.720 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:08.765 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:08.768+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:08.825 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.825 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:08.825 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:08.825 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:08.826 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:08.877 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:08.881+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:08.939 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:08.939 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:08.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:08.939 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:08.939 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:08.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:08.967 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:08.970+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:09.067 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:09.071+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:09.096 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.096 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:09.096 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:09.096 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.104 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:09.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:09.168 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:09.171+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:09.253 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.253 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:09.254 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:09.254 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:09.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.263 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:09.308 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:09.311+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:09.370 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:09.373+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:09.371 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.371 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:09.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:09.372 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:09.372 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.380 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:09.472 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:09.475+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:09.512 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":12,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:09.528 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:09.529 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:09.573 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:09.576+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:09.674 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:09.678+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:09.684 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.684 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:09.684 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:09.685 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:09.685 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:09.775 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:09.778+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:09.840 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:09.841 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:09.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:09.841 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:09.841 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:09.850 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:09.876 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:09.879+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:09.967 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:09.968 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:09.968 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:09.968 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:09.977 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:09.978 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:09.980+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:10.078 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:10.082+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:10.083 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.083 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:10.083 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:10.083 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:10.083 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.091 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:10.179 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:10.183+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:10.196 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.196 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:10.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:10.197 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:10.197 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.205 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:10.280 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:10.283+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:10.310 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.311 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:10.311 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:10.311 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.319 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:10.381 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:10.385+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:10.424 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.424 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:10.424 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:10.424 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:10.424 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.433 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:10.482 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:10.485+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:10.538 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.538 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:10.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:10.538 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:10.538 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:10.587 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:10.590+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:10.684 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:10.687+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:10.691 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.691 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:10.691 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:10.691 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.700 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:10.786 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:10.789+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:10.846 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.846 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:10.846 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:10.846 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:10.846 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.855 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:10.887 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:10.890+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:10.960 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:10.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:10.961 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:10.961 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:10.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:10.987 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:10.991+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:11.073 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:11.074 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:11.074 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:11.082 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:11.089 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:11.092+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:11.188 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:11.188 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:11.188 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:11.188 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:11.188 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:11.190 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:11.193+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:11.196 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:11.291 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:11.295+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.503509522000002, 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:11.304 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.310 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.470545930999997, 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:11.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.315 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.452781516000002, 2024-05-08T17:17:11.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.452688486, 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.424590461000001, 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.327 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.332 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.424480383999999, 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:11.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:11.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:11.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:11.337 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.400619855999999, 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.338 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.339 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.340 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.341 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.342 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.259260247999997, 2024-05-08T17:17:11.343 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.344 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.345 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.346 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.347 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.348 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.259143053000003, 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.349 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:11.350 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.351 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.352 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.353 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:11.354 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 37.259118102000002, 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.355 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:11.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:11.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:11.359 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861247243999998, 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:11.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:11.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:11.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:11.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861231664000002, 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.365 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:11.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:11.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:11.370 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861212344000002, 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.371 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.376 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861135840999999, 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:11.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:11.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861114246, 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:11.382 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.387 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861087851999997, 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.388 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:11.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:11.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:11.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.861013626000002, 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.394 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:11.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:11.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:11.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:11.399 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.375220128999999, 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:11.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:11.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:11.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:11.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.375206841999997, 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.405 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:11.410 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 36.375137774000002, 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:11.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:11.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:11.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:11.415 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:11.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:11.416 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:11.417 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:11.420+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:11.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:11.517 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:11.520+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:11.536 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:11.536 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:11.536 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:11.536 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:11.545 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:11.618 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:11.621+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:11.652 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:11.652 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:11.653 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:11.653 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:11.653 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:11.662 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:11.745 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:11.749+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:11.809 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 38.006617138999999, 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.810 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.811 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.812 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.812 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.812 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:11.812 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:11.812 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.813 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.814 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.814 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:11.814 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:11.814 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.814 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.815 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.815 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:11.815 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:11.816 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.816 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.816 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.816 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:11.816 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:11.817 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.817 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.817 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.817 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:11.818 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:11.819 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.820 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.820 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.820 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.820 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.820 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.984643986999998, 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.821 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.822 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.822 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.822 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.822 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:11.822 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.823 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.824 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.825 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.825 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:11.825 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:11.825 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.825 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.826 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.827 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:11.828 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:11.829 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.829 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.829 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.829 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:11.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:11.830 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:11.831 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.984187630999998, 2024-05-08T17:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.832 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.833 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:11.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:11.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.965145874999997, 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.840 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:11.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:11.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.844 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.956856815999998, 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.848 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.956368812000001, 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.850 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.955894379, 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.853 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.854 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:11.857 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.955501003999998, 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:11.861 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.949314567000002, 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:11.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:11.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:11.865 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.944511116000001, 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:11.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.869 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.941935657999998, 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:11.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:11.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:11.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.939149962000002, 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.934147338999999, 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.878 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.762971278000002, 2024-05-08T17:17:11.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:11.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.886 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.352387307999997, 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:11.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:11.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:11.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.879590196999999, 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:11.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:11.894 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.879004631000001, 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:11.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:11.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.878565223999999, 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:11.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.877917238999999, 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.903 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:11.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:11.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.872455512000002, 2024-05-08T17:17:11.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:11.911 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:11.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:11.913 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:11.916+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:12.014 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:12.017+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:12.061 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.061 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:12.061 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:12.061 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:12.115 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:12.118+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:12.174 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.174 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:12.174 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:12.175 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:12.175 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.183 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:12.224 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:12.228+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:12.317 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:12.321+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:12.329 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.330 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:12.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:12.330 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:12.330 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:12.418 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:12.422+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:12.446 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.446 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:12.446 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:12.446 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.454 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:12.527 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:12.539 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:12.542+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:12.602 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.602 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:12.602 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:12.602 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:12.603 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.612 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:12.620 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:12.623+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:12.717 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.717 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:12.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:12.717 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:12.717 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.720 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:12.724+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:12.726 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:12.821 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:12.825+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:12.873 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:12.873 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:12.873 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:12.874 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:12.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:12.924 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:12.927+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:13.023 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:13.027+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:13.027 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:13.027 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:13.027 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:13.036 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:13.125 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:13.128+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:13.135 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:17:13.135 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:17:13.135 INFO:tasks.thrashosds.thrasher:primary_affinity 2024-05-08T17:17:13.135 INFO:tasks.thrashosds.thrasher:Setting osd 4 primary_affinity to 1.000000 2024-05-08T17:17:13.136 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-08T17:17:13.142 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:13.142 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:13.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:13.142 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:13.142 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:13.152 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:13.226 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:13.229+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:13.270 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:13.270 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:13.270 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.279 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:13.363 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:13.366+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:13.425 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.426 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:13.426 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:13.426 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:13.426 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.428 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:13.432+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:13.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:13.529 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:13.533+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:13.580 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.581 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:13.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:13.581 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:13.581 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.589 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:13.630 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:13.633+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:13.693 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.693 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:13.693 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:13.693 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.701 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:13.731 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:13.734+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:13.806 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.806 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:13.806 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:13.806 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:13.806 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.815 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:13.859 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:13.862+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:13.920 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:13.920 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:13.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:13.920 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:13.920 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:13.929 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:13.970 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:13.973+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:14.033 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:14.036+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:14.074 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.074 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:14.075 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:14.075 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.083 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:14.167 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:14.170+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:14.228 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.228 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:14.228 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:14.228 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:14.228 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.235 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:14.239+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:14.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:14.337 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:14.340+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:14.381 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.382 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:14.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:14.382 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:14.382 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.390 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:14.438 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:14.441+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:14.496 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.496 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:14.496 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:14.496 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.505 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:14.539 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:14.542+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:14.639 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:14.642+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:14.650 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.650 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:14.650 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:14.651 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:14.651 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.659 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:14.743 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:14.746+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:14.807 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:14.807 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 41.006798922000002, 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:14.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:14.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:14.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:14.813 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.973835330999997, 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.814 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:14.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:14.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:14.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.819 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.956070916000002, 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:14.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:14.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:14.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.955977885999999, 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:14.825 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:14.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:14.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.927879861000001, 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:14.831 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.836 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.927769783999999, 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.837 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:14.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:14.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:14.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.903909255999999, 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.842 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.762549647999997, 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:14.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:14.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:14.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.762432453000002, 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:14.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:14.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.762407502000002, 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.859 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:14.864 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364536643999998, 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:14.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364521064000002, 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.870 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:14.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:14.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:14.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:14.876 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364501744000002, 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.877 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.882 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364425240999999, 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:14.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:14.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:14.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:14.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364403646, 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.888 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.893 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364377251999997, 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:14.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:14.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:14.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:14.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:14.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 40.364303026000002, 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.899 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:14.904 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.878509528999999, 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:14.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:14.909 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.878496241999997, 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.910 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:14.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:14.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:14.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:14.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.878427174000002, 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:14.915 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:14.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:14.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:14.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:14.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:14.920 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:14.921 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:14.922 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:14.922 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:14.925+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:15.023 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:15.027+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:15.041 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.041 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:15.041 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:15.041 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:15.134 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:15.138+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:15.197 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.197 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:15.197 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:15.197 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:15.197 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.206 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:15.248 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:15.251+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:15.327 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:15.330+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.551767376999997, 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.355 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:15.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.529794225000003, 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:15.359 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:15.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:15.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.529337869000003, 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.363 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:15.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:15.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:15.366 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.510296113000003, 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:15.367 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.368 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:15.369 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:15.370 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.371 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.502007054000003, 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.372 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.373 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.374 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:15.375 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.501519049999999, 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:15.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.377 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.378 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.501044616999998, 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.380 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:15.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.500651242000004, 2024-05-08T17:17:15.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.387 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:15.388 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.494464805, 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.389 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:15.390 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.391 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.392 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.489661353999999, 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.393 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:15.394 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:15.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.487085896000004, 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.397 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:15.401 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.4843002, 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:15.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.479297576999997, 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:15.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:15.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:15.408 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:15.409 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 41.308121516, 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.410 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.411 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.412 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.897537546000002, 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:15.416 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:15.417 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.424740434999997, 2024-05-08T17:17:15.418 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.419 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:15.420 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:15.421 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.422 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.424154868999999, 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.423715461999997, 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.427 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:15.429 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:15.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:15.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.423067477000004, 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:15.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.41760575, 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:15.436 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:15.440 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.442 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:15.442 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:15.445+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:15.543 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:15.546+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:15.592 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.592 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:15.592 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:15.592 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.601 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:15.645 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:15.646+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:15.744 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:15.748+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:15.747 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.747 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:15.747 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:15.747 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:15.747 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.756 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:15.845 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:15.849+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:15.903 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:15.903 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:15.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:15.903 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:15.903 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:15.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:15.953 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:15.956+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:16.048 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:16.051+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:16.057 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.057 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:16.057 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:16.057 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:16.148 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:16.152+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:16.171 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.171 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:16.171 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:16.171 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:16.171 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.180 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:16.249 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:16.253+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:16.287 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.287 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:16.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:16.287 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:16.287 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:16.350 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:16.353+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:16.442 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.443 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:16.443 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:16.443 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:16.452 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:16.454+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:16.552 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:16.556+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:16.598 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.598 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:16.598 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:16.598 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:16.598 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:16.690 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:16.694+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:16.754 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:16.837 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:16.838 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:16.838 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:16.758+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:16.839 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:16.883 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:16.886+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:16.956 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:16.956 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:16.956 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:16.956 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:16.957 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:16.960+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:16.965 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:17.057 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:17.060+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:17.111 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:17.112 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:17.112 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:17.112 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:17.112 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:17.121 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:17.133 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:17.133 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:17.151 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:17.151 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:17.165 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:17.168+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:17.168 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:17.168 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:17.186 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:17.187 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:17.203 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:17.204 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:17.221 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:17:17.221 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:17:17.227 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.227 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:17.238 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:17.238 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:17.255 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:17.255 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:17.258 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:17.262+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:17.360 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:17.363+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:17.380 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.380 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.389 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:17.460 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:17.463+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:17.493 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.494 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.502 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:17.561 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:17.565+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:17.648 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.648 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.657 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:17.662 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:17.666+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:17.763 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.763 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.763 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:17.766+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:17.772 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:17.864 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:17.867+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:17.877 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.877 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:17.886 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:17.965 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:17.968+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:17.992 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:17.992 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.000 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:18.085 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:18.089+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:18.118 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:17:18.119 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-08T17:17:18.149 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.149 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.156 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:18.198 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:18.202+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:18.268 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:18.272+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:18.303 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.303 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.312 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:18.396 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:18.400+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:18.458 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.459 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.467 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:18.508 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:18.512+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:18.570 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:18.573+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:18.585 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.585 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.594 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:18.670 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:18.674+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:18.743 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.743 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.753 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:18.797 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:18.801+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:18.860 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:18.860 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:18.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:18.904 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":16,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:18.904 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:18.907+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:18.917 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:17:18.918 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T17:17:18.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T17:17:18.960 INFO:tasks.ceph.mon.c:Started 2024-05-08T17:17:18.960 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:17:18.960 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:18.972 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:18.976+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:19.012 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:19.012 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:19.022 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:19.073 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:19.077+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:19.137 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:19.138 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:19.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:19.175 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:19.179+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:19.267 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:17:19.299 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:19.299 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:19.299 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:19.303+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:19.309 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:19.335 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":18,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:19.349 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:17:19.350 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T17:17:19.350 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:17:19.362 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T17:17:19.362 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:17:19.363 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:19.391 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:19.394+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:19.400 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:19.400 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:19.418 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:19.419 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:19.435 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:19.436 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:19.451 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:19.451 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:19.451 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:19.454 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:19.454 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:19.461 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:19.473 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:19.473 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:19.478 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:19.482+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:19.500 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:17:19.501 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:17:19.519 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:19.519 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:19.537 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:19.537 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:19.582 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:19.583 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:19.583 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:19.583 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:19.586+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:19.592 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:19.680 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:19.684+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:19.741 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:19.741 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:19.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:19.741 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:19.741 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:19.750 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:19.767 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":18,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:19.780 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:19.784+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:19.858 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:19.859 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:19.859 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:19.859 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:19.867 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:19.911 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:19.915+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:19.973 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:19.973 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:19.973 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:19.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:19.983 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:19.985+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:20.083 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:20.086+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:20.129 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:20.129 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:20.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:20.129 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:20.129 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:20.137 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:17:20.138 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:20.183 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:20.186+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:20.256 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.256 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:20.256 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:20.256 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.265 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:20.284 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:20.288+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:20.385 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:20.388+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:20.412 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:20.413 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.422 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:20.485 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:20.488+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:20.527 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.527 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:20.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:20.527 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:20.528 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:17:20.586 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:20.589+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:20.681 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:20.682 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:20.682 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.686 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:20.690+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:20.690 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_blocked_ops 2024-05-08T17:17:20.787 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:20.790+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:20.796 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.796 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:20.796 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:20.796 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:20.796 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.805 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_historic_ops 2024-05-08T17:17:20.890 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:20.893+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:20.952 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:20.953 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:20.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:20.988 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:20.992+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:21.090 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:21.093+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:21.106 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:21.106 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:21.106 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:21.106 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:21.115 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:21.191 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:21.194+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:21.221 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:21.221 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:21.221 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:21.221 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:21.221 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:21.231 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:21.315 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:21.318+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:21.376 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:21.377 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:21.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:21.377 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:21.377 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:21.385 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:21.393 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:21.397+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:21.494 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:21.497+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:21.533 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:21.533 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:21.533 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:21.542 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:21.549 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:17:21.629 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:21.630+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:21.685 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:21.685 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:21.688 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:21.688 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:21.688 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:21.688 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:21.688 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:21.695 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:21.699+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:21.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:21.702 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:21.702 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:21.719 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:21.719 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:21.735 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:21.735 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:21.752 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:21.752 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:21.769 INFO:teuthology.orchestra.run.smithi154.stdout:osd.5: {} 2024-05-08T17:17:21.769 INFO:teuthology.orchestra.run.smithi154.stderr:osd.5: osd_enable_op_tracker = '' 2024-05-08T17:17:21.786 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:21.787 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:21.796 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:21.799+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:21.805 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:21.805 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:21.844 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:21.844 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:21.853 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:21.900 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:21.904+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:21.963 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:21.963 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:21.972 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:21.997 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:22.000+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:22.098 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:22.101+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:22.119 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.119 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.127 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:22.211 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:22.215+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:22.274 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.274 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.282 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:22.323 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:22.327+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:22.401 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:22.405+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:22.428 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.428 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.437 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:22.521 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:22.525+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:22.583 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.583 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.592 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:22.634 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:22.637+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:22.703 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:22.707+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:22.739 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.740 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.749 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:22.782 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:22.804 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:17:22.807+0000 7fcb560a0640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 5 (PID: 32503) UID: 0 2024-05-08T17:17:22.898 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:22.898 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:22.904 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:22.908+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:22.907 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:23.005 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:23.009+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:23.055 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.056 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:23.107 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:23.110+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:23.136 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [] live_osds: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:17:23.136 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:17:23.136 INFO:tasks.thrashosds.thrasher:Killing osd 5, live_osds are [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:17:23.137 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T17:17:23.137 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:17:23.155 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":18,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"b","quorum_age":3,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:23.170 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T17:17:23.170 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.5 2024-05-08T17:17:23.170 INFO:tasks.thrashosds.thrasher:remote for osd 5 is ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:17:23.171 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-05-08T17:17:23.211 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:23.214+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:23.215 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.215 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.224 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:23.228 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T17:17:23.228 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-05-08T17:17:23.313 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:23.316+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:23.330 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.330 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.339 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:23.409 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:23.412+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:23.446 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.446 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.455 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:23.538 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:23.541+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:23.600 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.601 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.609 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:23.651 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:23.653+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:23.712 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:23.715+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:23.727 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.727 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:23.813 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:23.816+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:23.817 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:17:23.842 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.842 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:23.851 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:23.914 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:23.916+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:23.939 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:23.939 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:23.956 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:23.956 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:23.972 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:23.973 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:23.989 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:23.989 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:23.996 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:23.996 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.005 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:24.050 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:24.053+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:24.111 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.112 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.115 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:24.117+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:24.122 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:24.207 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:24.208 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:24.216 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:24.219+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:24.267 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.267 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.275 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:24.317 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:24.320+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:24.417 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:24.420+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:24.425 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.425 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.427 INFO:teuthology.orchestra.run.smithi183.stdout:2.3 2024-05-08T17:17:24.427 INFO:teuthology.orchestra.run.smithi183.stdout:2.2 2024-05-08T17:17:24.435 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:24.519 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:24.521+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:24.540 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.540 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.549 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:24.620 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:24.622+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:24.654 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.654 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:24.721 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:24.723+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:24.768 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:24.768 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:24.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:24.821 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:24.824+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:24.888 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:24.888 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:24.888 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:24.897 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:24.922 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:24.925+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:24.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 2.3 --file /var/log/ceph/exp.2.3.5 2024-05-08T17:17:25.003 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.004 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:25.004 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:25.004 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:25.004 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.012 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:25.056 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:25.059+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.316993081, 2024-05-08T17:17:25.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:25.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:25.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.124 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.295019928999999, 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:25.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.128 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.294563572999998, 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:25.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.132 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.275521816999998, 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:25.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:25.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.267232757999999, 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.137 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:25.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:25.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.266744754000001, 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.145 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.266270321, 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:25.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:25.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:25.150 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.265876945999999, 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.154 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.259690509000002, 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:25.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:25.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:25.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.254887058000001, 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:25.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:25.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.252311599999999, 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:25.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.249525904000002, 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.167 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:25.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.171 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.244523280999999, 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:25.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:25.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 51.073347220000002, 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.176 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:25.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:25.180 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.662763249999998, 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:25.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:25.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:25.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.189966138999999, 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:25.185 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:25.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:25.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:25.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:25.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:25.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:25.193 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.189380573000001, 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:25.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.188941165999999, 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.200 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:25.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:25.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.188293180999999, 2024-05-08T17:17:25.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:25.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:25.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:25.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.209 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.210 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 50.182831454000002, 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:25.211 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:25.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:25.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:25.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:25.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:25.217 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:25.219 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:25.221+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:25.319 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:25.322+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:25.367 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.367 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:25.367 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:25.367 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.375 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:25.420 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:25.423+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:25.521 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.521 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:25.521 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:25.521 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:25.521 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.521 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:25.523+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:25.529 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:25.621 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:25.624+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:25.676 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.676 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:25.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:25.676 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:25.676 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.684 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:25.725 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:25.728+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:25.824 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:25.827+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:25.831 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.831 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:25.831 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:25.831 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.840 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:25.925 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:25.928+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:25.947 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:25.947 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:25.947 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:25.947 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:25.947 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:25.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:26.039 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:26.042+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:26.074 INFO:teuthology.orchestra.run.smithi183.stderr:Exporting 2.3 info 2.3( empty local-lis/les=19/20 n=0 ec=15/15 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-05-08T17:17:26.074 INFO:teuthology.orchestra.run.smithi183.stderr:Export successful 2024-05-08T17:17:26.102 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:26.102 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:26.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:26.103 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:26.103 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:26.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:26.127 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:26.130+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:26.170 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:26.228 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:26.231+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:26.260 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:26.260 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:26.261 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:26.261 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:26.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:26.329 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:26.332+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:26.377 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:26.377 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:26.377 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:26.377 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:26.377 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:26.386 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:26.469 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:26.472+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:26.531 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:26.533+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:26.532 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:26.532 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:26.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:26.532 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:26.532 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:26.541 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:26.608 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 2.3 2024-05-08T17:17:26.652 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:26.655+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:26.657 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:26.657 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:26.667 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:26.733 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:26.736+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:26.813 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:26.813 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:26.813 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:26.814 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:26.814 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:26.822 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:26.834 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:26.837+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:26.928 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:26.928 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:26.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:26.928 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:26.928 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:26.934 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:26.937+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:26.938 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:27.036 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:27.038+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:27.052 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.052 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.061 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:27.136 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:27.139+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:27.167 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.167 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:27.237 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:27.240+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:27.280 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.280 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:27.338 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:27.341+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:27.395 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.395 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.404 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:27.445 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:27.448+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:27.539 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:27.542+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:27.548 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.548 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.557 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:27.641 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:27.644+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:27.705 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:27.705 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:27.710 INFO:teuthology.orchestra.run.smithi183.stdout: marking collection for removal 2024-05-08T17:17:27.710 INFO:teuthology.orchestra.run.smithi183.stdout:setting '_remove' omap key 2024-05-08T17:17:27.710 INFO:teuthology.orchestra.run.smithi183.stdout:finish_remove_pgs 2.3_head removing 2.3 2024-05-08T17:17:27.715 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:27.716 INFO:teuthology.orchestra.run.smithi183.stdout:Remove successful 2024-05-08T17:17:27.741 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:27.744+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:27.827 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:27.827 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:27.827 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:27.827 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:27.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:27.842 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:27.845+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:27.943 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:27.943 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:27.943 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:27.943 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:27.943 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:27.944 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:27.945+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:27.951 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:28.044 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:28.046+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.298912166000001, 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:28.102 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.106 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.276939014, 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:28.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:28.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:28.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.276482657999999, 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.257440901999999, 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.117 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:28.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.249151843, 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.123 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:28.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:28.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.248663839000002, 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.248189406000002, 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.132 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:28.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.136 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.247796031, 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.140 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.241609594000003, 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:28.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.236806143000003, 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.145 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:28.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:28.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:28.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.234230685, 2024-05-08T17:17:28.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:28.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:28.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:28.153 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.231444989000003, 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:28.157 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.226442366000001, 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:28.162 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.055266305000004, 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:28.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:28.168 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.644682334999999, 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:28.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:28.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.171885224, 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:28.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:28.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.171299658000002, 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:28.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:28.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:28.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.170860251000001, 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:28.182 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:28.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:28.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.170212266, 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:28.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:28.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 53.164750539000003, 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:28.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:28.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:28.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:28.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:28.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:28.196 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:28.199 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:28.201+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:28.226 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.3.5 2024-05-08T17:17:28.299 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:28.302+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:28.347 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.347 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:28.347 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:28.347 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.356 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:28.440 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:28.442+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:28.502 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.502 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:28.502 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:28.502 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:28.502 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.503 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:28.505+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:28.510 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:28.604 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:28.606+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:28.656 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:28.657 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:28.657 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:28.707 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:28.709+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:28.806 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:28.809+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:28.813 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.813 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:28.813 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:28.813 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.821 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:28.908 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:28.910+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:28.966 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:28.966 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:28.966 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:28.966 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:28.966 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:28.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:29.018 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:29.021+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:29.081 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:29.081 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:29.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:29.081 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:29.081 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:29.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:29.131 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:29.133+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:29.210 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:29.213+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:29.235 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:29.236 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:29.236 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:29.236 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:29.244 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:29.311 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:29.314+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:29.350 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:29.350 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:29.350 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:29.350 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:29.350 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:29.359 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:29.412 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:29.414+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:29.465 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:29.465 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:29.466 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:29.474 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:29.518 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:29.521+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:29.592 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:29.593 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:29.593 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:29.593 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:29.602 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:29.643 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:29.646+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:29.713 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:29.716+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:29.749 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:29.749 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:29.749 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:29.749 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:29.749 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:29.758 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:29.815 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:29.817+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:29.863 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:29.863 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:29.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:29.863 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:29.863 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:29.872 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:29.917 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:29.920+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:29.978 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:29.978 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:29.987 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:30.016 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:30.019+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:30.117 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:30.119+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:30.133 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:30.133 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:30.142 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:30.218 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:30.221+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:30.248 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:30.248 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:30.258 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:30.319 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:30.321+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:30.364 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:30.364 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:30.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:30.420 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:30.422+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:30.478 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:30.478 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:30.487 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:30.531 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:30.534+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:30.593 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:30.593 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:30.602 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:30.646 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:30.649+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:30.720 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:30.720 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:30.721 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:30.721 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:30.722 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:30.725+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:30.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:30.822 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:30.825+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:30.877 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:30.877 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:30.877 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:30.877 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:30.877 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:30.886 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:30.924 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:30.927+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:30.994 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.191549326999997, 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:30.995 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:30.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:30.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:30.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.169576175000003, 2024-05-08T17:17:30.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:31.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:31.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.169119819000002, 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.150078063000002, 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.008 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.012 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.141789004000003, 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.141300999999999, 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.017 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:31.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.140826566999998, 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.021 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:31.025 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.140433192000003, 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.029 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.134246754999999, 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.129443303999999, 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.038 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.126867846000003, 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:31.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.12408215, 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.043 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:31.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:31.047 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.119079526999997, 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.048 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:31.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.052 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.947903466, 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:31.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:31.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:31.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.537319496000002, 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.064522384999997, 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.061 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:31.065 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.063936818999998, 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:31.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:31.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:31.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.063497411999997, 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:31.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.062849427000003, 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.074 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 56.0573877, 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:31.078 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:31.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:31.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:31.083 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:31.083 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.084 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:31.087+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:31.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:31.185 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:31.188+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:31.190 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.190 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:31.191 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:31.191 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:31.286 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:31.288+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:31.304 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.305 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:31.305 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:31.305 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:31.305 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.313 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:31.387 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:31.390+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:31.460 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.460 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:31.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:31.460 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:31.460 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.468 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:31.512 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:31.515+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:31.575 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.575 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:31.575 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:31.576 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:31.589 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:31.592+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:31.691 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:31.694+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:31.730 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.731 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:31.731 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:31.731 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:31.731 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.739 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:31.791 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:31.794+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:31.844 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.845 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:31.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:31.845 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:31.845 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:31.853 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:31.892 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:31.895+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:31.993 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:31.996+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:31.999 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:31.999 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:31.999 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:31.999 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:32.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:32.094 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:32.096+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:32.113 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:32.113 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:32.114 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:32.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:32.206 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:32.209+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:32.268 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:32.268 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:32.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:32.268 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:32.268 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:32.277 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:32.321 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:32.324+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:32.394 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:32.394 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:32.394 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:32.394 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:32.397 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:32.400+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:32.403 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:32.498 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:32.501+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:32.550 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:32.550 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:32.550 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:32.560 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:32.601 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:32.604+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:32.701 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:32.703+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:32.705 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:32.705 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:32.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:32.705 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:32.705 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:32.715 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:32.802 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:32.805+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:32.862 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:32.862 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:32.871 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:32.903 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:32.906+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:33.005 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:33.008+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:33.019 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:33.019 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:33.028 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:33.106 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:33.109+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:33.134 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:33.134 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:33.143 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:33.207 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:33.210+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:33.287 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:33.287 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:33.296 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:33.341 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:33.344+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:33.401 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:33.401 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:33.408 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:33.411+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:33.410 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:33.510 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:33.512+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:33.556 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:33.556 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:33.565 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:33.610 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:33.613+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:33.683 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:33.683 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:33.683 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:33.683 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:33.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:33.734 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:33.736+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:33.813 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:33.815+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:33.840 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:33.840 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:33.840 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:33.840 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:33.840 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:33.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:33.914 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:33.917+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:33.996 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.193775193, 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:33.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:33.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:33.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.171802040999999, 2024-05-08T17:17:34.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:34.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:34.005 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.171345684999999, 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:34.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:34.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:34.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:34.009 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.152303928999999, 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:34.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.013 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.144014869999999, 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:34.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.143526866000002, 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:34.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:34.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.143052433000001, 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.142659058, 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:34.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.031 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.136472621000003, 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:34.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:34.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:34.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:34.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.131669170000002, 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.129093712, 2024-05-08T17:17:34.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:34.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:34.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.126308016000003, 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 60.121305393, 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:34.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:34.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.950129332000003, 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:34.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:34.057 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.539545361999998, 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.066748251, 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:34.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:34.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:34.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.065 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.066162685000002, 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:34.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.065723278, 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:34.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:34.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:34.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.065075293, 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:34.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:34.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:34.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 59.059613566000003, 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:34.078 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:34.082 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.084 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:34.084 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:34.087+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:34.185 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:34.187+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:34.190 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.190 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:34.190 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:34.190 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.199 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:34.285 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:34.289+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:34.345 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.345 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:34.346 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:34.346 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:34.346 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:34.387 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:34.390+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:34.460 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.460 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:34.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:34.460 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:34.460 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:34.487 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:34.490+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:34.575 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.575 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:34.575 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:34.575 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.584 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:34.627 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:34.630+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:34.690 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.690 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:34.690 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:34.690 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:34.690 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.690 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:34.692+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:34.699 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:34.790 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:34.794+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:34.845 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.845 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:34.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:34.845 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:34.845 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.853 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:34.897 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:34.900+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:34.960 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:34.960 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:34.960 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:34.960 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:34.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:35.010 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:35.012+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:35.094 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:35.096+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:35.114 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:35.114 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:35.115 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:35.115 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:35.115 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:35.123 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:35.194 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:35.197+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:35.229 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:35.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:35.229 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:35.229 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:35.237 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:35.295 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:35.298+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:35.356 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:35.356 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:35.356 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:35.365 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:35.396 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:35.399+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:35.497 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:35.500+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:35.511 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:35.512 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:35.512 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:35.512 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:35.512 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:35.521 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:35.599 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:35.602+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:35.666 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:35.666 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:35.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:35.667 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:35.667 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:35.676 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:35.700 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:35.703+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:35.781 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:35.781 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:35.790 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:35.835 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:35.838+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:35.895 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:35.895 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:35.901 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:35.904+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:35.904 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:36.002 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:36.004+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:36.049 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:36.049 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:36.058 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:36.103 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:36.106+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:36.163 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:36.163 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:36.173 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:36.217 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:36.220+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:36.278 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:36.279 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:36.287 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:36.305 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:36.308+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:36.406 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:36.409+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:36.433 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:36.433 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:36.442 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:36.506 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:36.509+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:36.559 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:36.559 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:36.560 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:36.560 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:36.568 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:36.608 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:36.611+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:36.709 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:36.711+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:36.715 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:36.715 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:36.715 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:36.724 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:36.810 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:36.813+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:36.872 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.069365267000002, 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:36.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.876 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.047392115000001, 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:36.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:36.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.881 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.046935759, 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.027894003, 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:36.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:36.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:36.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.019604944000001, 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:36.894 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.019116940000004, 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.898 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.018642507000003, 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:36.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:36.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:36.902 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.018249132000001, 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.012062694999997, 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.907 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:36.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.007259243999997, 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.911 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:36.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:36.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.004683786000001, 2024-05-08T17:17:36.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:36.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.919 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.001898089999997, 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:36.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:36.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.923 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.996895467000002, 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:36.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.825719405999997, 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 62.415135436, 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.942338325000001, 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.941752759000003, 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.941313352000002, 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.940665367000001, 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:36.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:36.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 61.935203639999997, 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:36.952 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:36.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:36.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:36.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:36.956 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:36.957 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:36.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:36.959 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:36.961+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:37.059 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:37.061+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:37.064 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.064 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:37.064 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:37.064 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.073 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:37.159 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:37.162+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:37.220 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.220 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:37.220 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:37.220 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:37.220 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:37.271 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:37.273+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:37.361 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:37.364+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:37.376 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.376 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:37.376 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:37.376 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:37.376 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.385 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:37.469 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:37.472+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:37.533 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.533 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:37.533 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:37.533 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:37.566 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:37.566+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:37.664 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:37.667+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:37.688 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.688 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:37.688 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:37.688 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:37.688 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.697 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:37.764 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:37.767+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:37.803 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.803 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:37.803 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:37.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:37.803 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:37.866 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:37.869+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:37.958 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:37.958 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:37.958 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:37.958 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:37.967 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:37.967 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:37.969+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:38.067 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:38.070+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:38.072 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:38.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:38.168 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:38.171+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:38.227 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:38.227 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:38.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:38.228 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:38.228 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:38.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:38.278 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:38.280+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:38.354 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:38.354 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:38.354 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:38.363 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:38.407 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:38.410+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:38.469 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:38.469 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:38.469 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:38.469 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:38.470 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:38.471 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:38.474+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:38.478 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:38.571 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:38.575+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:38.624 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:38.625 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:38.625 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:38.634 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:38.672 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:38.675+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:38.739 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:38.739 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:38.748 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:38.773 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:38.775+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:38.853 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:38.853 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:38.862 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:38.874 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:38.878+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:38.967 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:38.967 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:38.975 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:38.978+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:38.976 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:39.078 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:39.081+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:39.122 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:39.122 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:39.131 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:39.179 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:39.182+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:39.277 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:39.277 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:39.283 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:39.286+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:39.286 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:39.384 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:39.387+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:39.391 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:39.391 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:39.400 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:39.485 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:39.488+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:39.519 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:39.519 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:39.519 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:39.519 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:39.528 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:39.611 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:39.614+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:39.675 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:39.675 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:39.675 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:39.675 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:39.675 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:39.684 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:39.726 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:39.728+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:39.788 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:39.791+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:39.832 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.029773176000006, 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.833 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.834 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.835 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.836 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.007800024000005, 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.837 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:39.838 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:39.839 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.840 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.841 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.007343668000004, 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:39.842 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:39.843 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.845 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.988301911999997, 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:39.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:39.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:39.848 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:39.849 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.980012853000005, 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.850 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.851 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.852 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.853 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.979524849000001, 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.854 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:39.855 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.856 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.857 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.979050416000007, 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.858 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:39.859 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:39.860 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.861 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:39.862 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.978657041000005, 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.863 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.864 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.865 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.866 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.972470603999994, 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:39.867 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:39.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:39.869 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:39.870 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.967667152999994, 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.871 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.872 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.873 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.874 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.965091694999998, 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.875 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.876 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.962305998999994, 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.957303375999999, 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.786127315000002, 2024-05-08T17:17:39.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.889 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.890 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 65.375543344999997, 2024-05-08T17:17:39.891 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.892 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.902746234000006, 2024-05-08T17:17:39.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.897 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:39.899 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.902160667999993, 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:39.903 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:39.905 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.901721261000006, 2024-05-08T17:17:39.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:39.905 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.905 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:39.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:39.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.901073276000005, 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.909 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:39.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:39.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:39.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.895611548999995, 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:39.913 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:39.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:39.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:39.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:39.917 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:39.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:39.919 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:39.922+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:40.019 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:40.023+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:40.026 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.026 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.026 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:40.027 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.035 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:40.120 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.123+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.140 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.140 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.140 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:40.140 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:40.141 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:40.221 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.224+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.254 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.254 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:40.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:40.254 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:40.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:40.322 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.325+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.368 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.368 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.368 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:40.368 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:40.423 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.426+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.482 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.483 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.483 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:40.483 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:40.483 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.492 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:40.523 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:40.526+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:40.597 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.597 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:40.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:40.597 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:40.597 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:40.624 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.627+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.712 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.712 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.712 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:40.712 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.723 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:40.724 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:40.727+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:40.825 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:40.828+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:40.827 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:40.827 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:40.827 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.836 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:40.930 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:40.933+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:40.982 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:40.982 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:40.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:40.982 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:40.982 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:40.991 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:41.035 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:41.038+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:41.108 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:41.108 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:41.108 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:41.108 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:41.117 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:41.135 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:41.138+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:41.235 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:41.238+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:41.263 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:41.263 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:41.263 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:41.263 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:41.263 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:41.272 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:41.338 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:41.340+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:41.418 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:41.418 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:41.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:41.418 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:41.418 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:41.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:41.471 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:41.474+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:41.531 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:41.532 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:41.538 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:41.541+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:41.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:41.639 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:41.642+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:41.688 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:41.688 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:41.697 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:41.740 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:41.743+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:41.803 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:41.803 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:41.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:41.841 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:41.845+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:41.918 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:41.918 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:41.926 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:41.942 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:41.945+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:42.033 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:42.033 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:42.042 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:42.043 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:42.046+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:42.144 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:42.147+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:42.147 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:42.147 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:42.157 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:42.247 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:42.249+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:42.275 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:42.275 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:42.275 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:42.275 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:42.284 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:42.347 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:42.351+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:42.392 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:42.392 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:42.392 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:42.392 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:42.392 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:42.401 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:42.485 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:42.489+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:42.549 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:42.549 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.746148332999994, 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:42.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.554 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.724175181000007, 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:42.559 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.723718825000006, 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:42.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.704677068999999, 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:42.564 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.696388010000007, 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.569 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:42.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:42.573 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.695900006000002, 2024-05-08T17:17:42.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:42.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.695425572999994, 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:42.579 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:42.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:42.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.695032198000007, 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.584 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:42.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:42.588 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.688845760999996, 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:42.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.684042309999995, 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:42.594 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.681466852, 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.599 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:42.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.678681155999996, 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.604 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:42.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:42.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.673678533, 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.609 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:42.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:42.613 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.502502472000003, 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:42.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 68.091918501999999, 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:42.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.623 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 67.619121390999993, 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:42.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:42.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:42.628 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 67.618535824999995, 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:42.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:42.633 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 67.618096417999993, 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:42.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:42.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 67.617448433000007, 2024-05-08T17:17:42.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:42.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:42.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 67.611986705999996, 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:42.643 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:42.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:42.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:42.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:42.648 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:42.649 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:42.552+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:42.650 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:42.650 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:42.653+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:42.751 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:42.754+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:42.798 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:42.798 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:42.798 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:42.798 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:42.807 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:42.890 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:42.894+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:42.952 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:42.953 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:42.953 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:42.953 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:42.956+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:42.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:43.054 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:43.057+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:43.108 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.108 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:43.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:43.108 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:43.108 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:43.157 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:43.161+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:43.256 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:43.259+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:43.262 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.263 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:43.263 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:43.263 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:43.357 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:43.360+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:43.421 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.429 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:43.473 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:43.477+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:43.536 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.536 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:43.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:43.537 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:43.537 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.545 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:43.587 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:43.590+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:43.661 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:43.664+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:43.691 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.691 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:43.691 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:43.691 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:43.762 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:43.764+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:43.806 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.806 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:43.806 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:43.806 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:43.806 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.815 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:43.827 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:43.831+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:43.863 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:43.866+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:43.919 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:43.920 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:43.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:43.920 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:43.920 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:43.929 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:43.973 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:43.976+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:44.045 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:44.045 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:44.045 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:44.056 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:44.097 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:44.100+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:44.166 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:44.169+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:44.201 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:44.201 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:44.201 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:44.201 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:44.201 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:44.211 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:44.267 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:44.270+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:44.316 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:44.316 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:44.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:44.316 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:44.316 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:44.325 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:44.367 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:44.371+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:44.430 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:44.431 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:44.440 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:44.484 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:44.487+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:44.546 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:44.546 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:44.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:44.570 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:44.574+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:44.671 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:44.675+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:44.701 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:44.701 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:44.710 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:44.773 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:44.776+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:44.815 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:44.815 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:44.824 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:44.852 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:44.854+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:44.873 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:44.877+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:44.930 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:44.930 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:44.939 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:44.980 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:44.983+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:45.048 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:45.051+0000 7f78bb525640 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.019302+0000 front 2024-05-08T17:17:23.019286+0000 (oldest deadline 2024-05-08T17:17:44.119084+0000) 2024-05-08T17:17:45.075 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:45.079+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:45.085 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:45.085 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:45.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:45.176 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:45.179+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:45.205 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:45.205 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:45.205 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:45.205 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:45.214 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:45.278 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:45.281+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:45.319 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:45.320 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:45.320 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:45.320 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:45.320 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:45.329 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:45.379 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:45.382+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:45.477 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:45.477 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:45.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:45.477 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.674233280999999, 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.478 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:45.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.485 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.652260128999998, 2024-05-08T17:17:45.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:45.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:45.492 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.651803772999997, 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:45.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:45.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.497 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.632762017000005, 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:45.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:45.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:45.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.624472957999998, 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:45.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:45.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:45.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.623984953999994, 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:45.507 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:45.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.623510521, 2024-05-08T17:17:45.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:45.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.623117145999998, 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:45.519 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.616930709000002, 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.612127258000001, 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.609551800000006, 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:45.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:45.541 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.606766104000002, 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:45.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:45.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:45.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.601763481000006, 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:45.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.551 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.430587419999995, 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.552 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:45.556 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 71.020003450000004, 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:45.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:45.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 70.547206338999999, 2024-05-08T17:17:45.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:45.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:45.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 70.546620773000001, 2024-05-08T17:17:45.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 70.546181365999999, 2024-05-08T17:17:45.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:45.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:45.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 70.545533380999998, 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.578 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:45.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:45.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:45.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.584 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 70.540071654000002, 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:45.585 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:45.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:45.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:45.588 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:45.589 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:45.591 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:45.591 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:45.594+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:45.617 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:45.620+0000 7f6df2e84640 -1 osd.7 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.981524+0000 front 2024-05-08T17:17:22.981541+0000 (oldest deadline 2024-05-08T17:17:44.681439+0000) 2024-05-08T17:17:45.691 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:45.695+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:45.697 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:45.697 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:45.697 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:45.706 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:45.768 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:45.771+0000 7ff98514d640 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:19.418570+0000 front 2024-05-08T17:17:19.418535+0000 (oldest deadline 2024-05-08T17:17:45.318355+0000) 2024-05-08T17:17:45.792 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:45.796+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:45.854 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:45.854 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:45.854 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:45.854 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:45.862 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:45.863 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:45.865+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:45.893 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:45.896+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:45.994 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:45.998+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:46.008 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.008 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:46.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:46.008 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:46.008 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.009 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:46.012+0000 7f61444ab640 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.200470+0000 front 2024-05-08T17:17:22.200441+0000 (oldest deadline 2024-05-08T17:17:45.700416+0000) 2024-05-08T17:17:46.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:46.028 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:46.032+0000 7f78bb525640 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.019302+0000 front 2024-05-08T17:17:23.019286+0000 (oldest deadline 2024-05-08T17:17:44.119084+0000) 2024-05-08T17:17:46.096 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:46.099+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:46.133 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.134 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:46.134 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:46.134 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:46.196 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:46.199+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:46.288 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.288 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:46.288 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:46.288 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:46.289 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.297 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:46.298 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:46.300+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:46.399 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:46.402+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:46.402 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.402 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:46.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:46.402 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:46.402 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.411 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:46.499 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:46.502+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:46.558 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.558 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:46.559 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:46.559 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:46.586 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:46.589+0000 7f6df2e84640 -1 osd.7 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.981524+0000 front 2024-05-08T17:17:22.981541+0000 (oldest deadline 2024-05-08T17:17:44.681439+0000) 2024-05-08T17:17:46.601 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:46.603+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:46.673 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.674 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:46.674 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:46.674 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:46.674 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:46.701 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:46.704+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:46.788 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:46.788 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:46.788 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:46.788 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:46.788 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:46.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:46.802 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:46.806+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:46.805 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:46.809+0000 7ff98514d640 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:19.418570+0000 front 2024-05-08T17:17:19.418535+0000 (oldest deadline 2024-05-08T17:17:45.318355+0000) 2024-05-08T17:17:46.819 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:46.822+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:46.904 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:46.906+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:46.916 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:46.916 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:46.916 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:46.916 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:46.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:46.982 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:46.985+0000 7f61444ab640 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.200470+0000 front 2024-05-08T17:17:22.200441+0000 (oldest deadline 2024-05-08T17:17:45.700416+0000) 2024-05-08T17:17:47.010 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:47.013+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:47.064 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:47.068+0000 7f78bb525640 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.019302+0000 front 2024-05-08T17:17:23.019286+0000 (oldest deadline 2024-05-08T17:17:44.119084+0000) 2024-05-08T17:17:47.072 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:47.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:47.072 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:47.072 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:47.072 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:47.082 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:47.106 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:47.109+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:47.207 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:47.210+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:47.227 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:47.228 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:47.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:47.228 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:47.228 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:47.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:47.308 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:47.311+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:47.342 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:47.342 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:47.351 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:47.410 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:47.413+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:47.496 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:47.496 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:47.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:47.510 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:47.514+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:47.578 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:47.581+0000 7f6df2e84640 -1 osd.7 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.981524+0000 front 2024-05-08T17:17:22.981541+0000 (oldest deadline 2024-05-08T17:17:44.681439+0000) 2024-05-08T17:17:47.610 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:47.610 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:47.611 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:47.614+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:47.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:47.712 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:47.715+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:47.765 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:47.765 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:47.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:47.776 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:47.778+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:47.816 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:47.819+0000 7f3920f77640 -1 osd.2 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:21.918450+0000 front 2024-05-08T17:17:21.918392+0000 (oldest deadline 2024-05-08T17:17:47.218299+0000) 2024-05-08T17:17:47.819 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:47.822+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:47.839 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:47.842+0000 7ff98514d640 -1 osd.1 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:19.418570+0000 front 2024-05-08T17:17:19.418535+0000 (oldest deadline 2024-05-08T17:17:45.318355+0000) 2024-05-08T17:17:47.881 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:47.881 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:47.889 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:47.914 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:47.917+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:47.945 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:47.949+0000 7f61444ab640 -1 osd.3 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.200470+0000 front 2024-05-08T17:17:22.200441+0000 (oldest deadline 2024-05-08T17:17:45.700416+0000) 2024-05-08T17:17:48.015 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:48.018+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:48.035 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:48.035 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:48.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:48.099 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:48.102+0000 7f78bb525640 -1 osd.0 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.019302+0000 front 2024-05-08T17:17:23.019286+0000 (oldest deadline 2024-05-08T17:17:44.119084+0000) 2024-05-08T17:17:48.116 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:48.119+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:48.163 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:48.163 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:48.163 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:48.163 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:48.172 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:48.256 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:48.260+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:48.318 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:48.321+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:48.321 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:48.321 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:48.321 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:48.321 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:48.321 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:48.331 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:17:48.332 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:48.429 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:48.433+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:48.494 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.691111004999996, 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:48.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:48.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:48.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.669137852999995, 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:48.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:48.502 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.668681496999994, 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:48.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:48.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:48.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.649639741000001, 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:48.510 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.641350681999995, 2024-05-08T17:17:48.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:48.518 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.640862678000005, 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:48.522 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.640388244999997, 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:48.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:48.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.639994869999995, 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.633808432999999, 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:48.534 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.629004981999998, 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:48.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:48.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.626429524000002, 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:48.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.623643827999999, 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.543 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:48.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:48.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:48.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.618641205000003, 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.547 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:48.551 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.447465144000006, 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:48.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:48.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:48.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 74.036881174000001, 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 73.564084062999996, 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:48.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:48.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:48.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 73.563498496999998, 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 73.563059089999996, 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.568 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:48.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:48.570 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:48.571 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 73.562411104999995, 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.572 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 73.556949377999999, 2024-05-08T17:17:48.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:48.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:48.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.580 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.581 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:48.581 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:48.576+0000 7f6df2e84640 -1 osd.7 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.981524+0000 front 2024-05-08T17:17:22.981541+0000 (oldest deadline 2024-05-08T17:17:44.681439+0000) 2024-05-08T17:17:48.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:48.583 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:48.585+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:48.682 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:48.686+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:48.731 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:48.731 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:48.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.731 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:17:48.731 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 0.45273808500000001, 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.452758037, 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "waiting for readable", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:48.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:17:48.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:48.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 1 2024-05-08T17:17:48.735 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:48.740 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:48.765 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:48.769+0000 7f0ca224d640 -1 osd.6 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:20.377076+0000 front 2024-05-08T17:17:20.376967+0000 (oldest deadline 2024-05-08T17:17:43.276916+0000) 2024-05-08T17:17:48.783 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:48.787+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:48.807 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:48.811+0000 7f3920f77640 -1 osd.2 23 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:21.918450+0000 front 2024-05-08T17:17:21.918392+0000 (oldest deadline 2024-05-08T17:17:47.218299+0000) 2024-05-08T17:17:48.837 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:48.840+0000 7ff98514d640 -1 osd.1 23 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:19.418570+0000 front 2024-05-08T17:17:19.418535+0000 (oldest deadline 2024-05-08T17:17:45.318355+0000) 2024-05-08T17:17:48.884 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:48.888+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:48.887 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:48.887 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:48.887 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:48.887 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:48.887 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:48.896 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:48.903 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:48.906+0000 7f61444ab640 -1 osd.3 23 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.200470+0000 front 2024-05-08T17:17:22.200441+0000 (oldest deadline 2024-05-08T17:17:45.700416+0000) 2024-05-08T17:17:48.985 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:48.988+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:49.009 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.009 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:49.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:49.009 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:49.009 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.018 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:49.076 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:49.079+0000 7f78bb525640 -1 osd.0 23 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.019302+0000 front 2024-05-08T17:17:23.019286+0000 (oldest deadline 2024-05-08T17:17:44.119084+0000) 2024-05-08T17:17:49.086 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:49.090+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:49.164 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.165 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:49.165 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:49.165 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.174 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:49.188 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:49.191+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:49.283 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:49.286+0000 7f5a2d43d640 -1 osd.4 22 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:23.112854+0000 front 2024-05-08T17:17:23.112816+0000 (oldest deadline 2024-05-08T17:17:48.412674+0000) 2024-05-08T17:17:49.288 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:49.292+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:49.321 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.321 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:49.321 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:49.321 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:49.321 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.330 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:49.336 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:17:49.355 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:49.355 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:49.373 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:49.373 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:49.376 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:17:49.376 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:17:49.389 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:49.393+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:49.443 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.164395394, 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:49.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:17:49.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:49.447 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:49.448 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:49.490 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:49.494+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:49.558 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.558 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:49.558 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:49.558 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:49.574 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:49.577+0000 7f6df2e84640 -1 osd.7 23 heartbeat_check: no reply from 172.21.15.183:6805 osd.5 since back 2024-05-08T17:17:22.981524+0000 front 2024-05-08T17:17:22.981541+0000 (oldest deadline 2024-05-08T17:17:44.681439+0000) 2024-05-08T17:17:49.611 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:49.615+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:49.673 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:49.673 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:49.692 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:49.695+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:49.793 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:49.797+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:49.828 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:49.828 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:49.829 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:49.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:49.829 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:49.838 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:49.893 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:49.897+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:49.946 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:49.947 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:49.947 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:49.947 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:49.955 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:49.995 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:49.998+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:50.059 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.059 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:50.060 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:50.060 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:50.060 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:50.113 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:50.116+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:50.174 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.174 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:50.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:50.174 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:50.174 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.183 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:50.224 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:50.227+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:50.297 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:50.300+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:50.328 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.328 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:50.328 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:50.328 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.337 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:50.346 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:17:50.386 INFO:teuthology.orchestra.run.smithi183.stdout:get_pg_num_history pg_num_history pg_num_history(e22 pg_nums {1={14=1},2={15=8}} deleted_pools ) 2024-05-08T17:17:50.386 INFO:teuthology.orchestra.run.smithi183.stdout:Importing pgid 2.3 2024-05-08T17:17:50.386 INFO:teuthology.orchestra.run.smithi183.stdout:write_pg epoch 20 info 2.3( empty local-lis/les=19/20 n=0 ec=15/15 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-05-08T17:17:50.391 INFO:teuthology.orchestra.run.smithi183.stdout:Import successful 2024-05-08T17:17:50.398 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:50.402+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:50.483 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.483 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:50.483 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:50.483 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:50.483 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.492 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:50.534 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:50.536+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:50.565 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:50.579 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:17:50.579 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T17:17:50.579 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T17:17:50.581 INFO:tasks.ceph.mon.b:Started 2024-05-08T17:17:50.582 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:17:50.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:50.600 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:50.603+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:50.640 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.640 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:50.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:50.640 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:50.640 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.650 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:50.701 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:50.705+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:50.795 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:50.796 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.803 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:50.806+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:50.804 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:50.904 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:50.907+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:50.911 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /var/log/ceph/exp.2.3.5 2024-05-08T17:17:50.952 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:50.952 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:50.964 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:50.972 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:51.026 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:51.030+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:51.106 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:51.109+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.323108167000001, 2024-05-08T17:17:51.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:51.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:51.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:51.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:51.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:51.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:51.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:51.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.290144576000003, 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:51.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:51.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:51.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.272380161000001, 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:51.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:51.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:51.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:51.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.272287130999999, 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.149 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:51.155 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.244189105999993, 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:51.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:51.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:51.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:51.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:51.162 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.244079029000005, 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:51.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.220218501000005, 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:51.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:51.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.174 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.078858893000003, 2024-05-08T17:17:51.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:51.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.180 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.078741698000002, 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:51.181 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:51.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:51.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 77.078716747000001, 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:51.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:51.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:51.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:51.193 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680845888999997, 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.194 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:51.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:51.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:51.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:51.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:51.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.204 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.204 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:51.204 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680830309000001, 2024-05-08T17:17:51.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.205 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:51.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.206 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:51.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:51.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:51.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:51.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:51.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:51.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.213 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:51.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:51.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:51.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:51.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:51.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:51.217 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680810988999994, 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.220 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.220 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:51.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.220 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:51.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:51.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:51.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:51.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:51.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:51.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:51.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:51.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:51.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:51.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:51.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:51.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:51.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:51.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:51.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:51.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:51.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.234 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680734486000006, 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:51.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:51.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:51.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:51.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:51.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:51.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.248 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680712890999999, 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.249 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.250 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:51.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:51.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:51.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:51.262 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.263 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680686496999996, 2024-05-08T17:17:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.264 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:51.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:51.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:51.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.680612271000001, 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:51.278 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.194818773999998, 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:51.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:51.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:51.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:51.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.194805486999996, 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.284 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.288 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.194736418999994, 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:51.289 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:51.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:51.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:51.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:51.294 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:51.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:51.297 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:51.299+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:51.337 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:17:51.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:17:51.408 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:51.408 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:51.408 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:51.408 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:51.408 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:51.411+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:51.417 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:51.501 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:51.505+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:51.517 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:51.517 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:51.535 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:51.535 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:51.567 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:51.568 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:51.593 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:51.594 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:51.598 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:51.602+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:51.601 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:51.601 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:51.605 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:51.619 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:51.620 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:51.620 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:17:51.636 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:51.637 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:51.654 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:51.654 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:51.658 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:17:51.658 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:17:51.699 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:51.703+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:51.725 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:51.726 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:51.735 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:51.799 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:51.803+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:51.881 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:51.881 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:51.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:51.900 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:51.904+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:51.995 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:51.995 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.001 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:52.004+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:52.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:52.102 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:52.106+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:52.151 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.151 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:52.204 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:52.207+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:52.304 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:52.308+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:52.306 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.306 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:52.405 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:52.409+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:52.422 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.423 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.432 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:52.506 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:52.509+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:52.578 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.578 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.588 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:52.631 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:52.635+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:52.694 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.694 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.703 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:52.708 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:52.712+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:52.809 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:52.812+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:52.849 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:52.849 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:52.858 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:52.909 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:52.912+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:53.006 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.006 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.011 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:53.014+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:53.015 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:53.112 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:53.115+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:53.133 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.133 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.142 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:53.215 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:53.217+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:53.247 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.248 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:53.315 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:53.319+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:53.353 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:17:53.403 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.403 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.412 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:53.420 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:53.424+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:53.517 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.517 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.521 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:53.525+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:53.526 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:53.623 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:53.626+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:53.631 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.631 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.640 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:53.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:17:53.724 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:53.727+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:53.746 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:53.746 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:53.755 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:53.783 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:53.783 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:53.800 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:53.800 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:53.817 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:53.817 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:53.826 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:53.830+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:53.834 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:53.834 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:53.851 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:53.851 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:53.851 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:17:53.869 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:53.869 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:53.886 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:53.886 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:53.889 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:17:53.890 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:17:53.902 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:53.903 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:53.912 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:53.955 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:53.958+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:53.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:54.017 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:54.018 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:54.018 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:54.018 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:54.018 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:54.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:54.030 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:54.034+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:54.131 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:54.134+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:54.173 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:54.173 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:54.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:54.173 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:54.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.372372576000004, 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.174 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:54.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.339408985000006, 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.321644570000004, 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.186 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:54.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:54.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:54.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.321551540000002, 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.191 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:54.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:54.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:54.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.293453514999996, 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.293343437999994, 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.202 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.203 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:54.204 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.205 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:54.206 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.207 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.269482909999994, 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.208 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:54.209 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:54.210 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:54.211 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:54.212 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.128123302000006, 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.213 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.214 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.215 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.216 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.217 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.128006107000004, 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:54.218 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.219 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:17:54.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:17:54.221 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:17:54.222 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 80.127981156000004, 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:17:54.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.730110298, 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.229 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:17:54.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.730094718000004, 2024-05-08T17:17:54.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.237 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.238 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.239 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.730075397999997, 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:17:54.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:17:54.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:17:54.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.729998894999994, 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:17:54.245 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.729977300000002, 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.251 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:17:54.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:17:54.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.729950905999999, 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:17:54.256 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:17:54.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.261 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.729876680000004, 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.244083183000001, 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:54.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.244069895999999, 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.244000827999997, 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:17:54.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:17:54.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:54.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:54.283 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:54.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:54.285 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:54.288+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:54.345 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":6,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:54.356 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:17:54.396 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:54.396 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:54.396 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:54.397 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:54.397 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:54.400+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:54.405 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:54.486 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:54.490+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:54.551 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:54.551 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:54.551 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:54.551 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:54.551 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:54.560 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:54.604 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:54.608+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.668 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.865022955000001, 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:17:54.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:17:54.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:17:54.672 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.843049803, 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.842593446999999, 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.677 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:17:54.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:17:54.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:17:54.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.823551691000006, 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:17:54.681 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:17:54.685 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.815262632, 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:54.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:17:54.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.814774627999995, 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:17:54.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:17:54.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:17:54.693 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.814300195000001, 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.81390682, 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.698 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:17:54.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:54.702 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.807720383000003, 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:17:54.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:17:54.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.802916932000002, 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:17:54.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.800341474000007, 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.797555778000003, 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.715 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:17:54.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:17:54.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:17:54.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.792553154999993, 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.719 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:54.723 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.621377093999996, 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:17:54.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:17:54.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:17:54.727 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.210793124000006, 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:17:54.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:17:54.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.737996013, 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:17:54.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:17:54.735 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.737410447000002, 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.73697104, 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:17:54.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:17:54.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.736323055, 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.744 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:17:54.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:17:54.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:17:54.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.748 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.730861328000003, 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:17:54.752 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.753 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.753 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.753 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:54.753 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:54.753 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:54.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:54.755 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:54.757+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:54.855 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:54.858+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:54.903 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:54.904 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:54.904 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:54.904 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:54.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:54.956 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:54.959+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:55.056 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:55.060+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:55.058 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.058 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:55.058 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:55.058 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:55.058 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.067 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:55.158 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:55.161+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:17:55.172 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 6.8936576809999996, 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:55.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:17:55.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:17:55.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:17:55.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:17:55.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.178 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:55.179 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:55.258 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:55.262+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:55.286 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.287 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:55.287 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:55.287 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.295 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:55.359 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:55.363+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:55.441 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:55.442 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:55.442 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.451 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:55.460 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:55.464+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 7.2772889950000001, 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:55.556 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:55.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:55.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:55.560 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.561 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:55.565+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:55.564 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:55.662 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:55.666+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:55.670 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.670 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:55.670 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:55.670 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:55.763 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:55.767+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:55.825 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:55.825 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:55.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:55.826 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.835 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:55.878 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:55.879+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:55.890 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:17:55.965 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:55.969+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:55.978 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:17:55.978 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:17:55.978 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T17:17:55.978 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:17:55.978 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:17:55.979 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T17:17:55.985 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:55.985 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:55.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:55.985 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:55.985 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:55.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:56.049 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:56.049 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:56.066 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:56.070+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:56.067 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:56.067 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:56.084 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:56.084 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:56.089 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:56.089 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T17:17:56.089 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:56.101 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:56.101 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:56.112 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:56.113 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:56.113 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:56.118 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:56.119 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:56.119 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:17:56.122 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:56.136 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:56.136 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:56.153 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:56.153 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:56.156 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:17:56.157 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:17:56.167 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:56.171+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:56.227 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.227 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:56.277 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:56.281+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:56.368 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:56.372+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:56.372 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:17:56.381 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.381 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.390 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:56.470 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:56.473+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:56.535 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.535 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.544 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:56.589 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:56.593+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:56.649 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.649 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.658 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:56.700 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:56.703+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:56.772 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:56.776+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:56.805 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.805 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.814 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:56.898 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:56.902+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:56.960 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:56.961 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:56.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:56.974 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:56.977+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:57.074 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:57.078+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:57.117 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.117 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.126 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:57.175 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:57.179+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:57.231 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.231 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:17:57.284 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:57.288+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:57.354 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.354 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.360 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:17:57.364 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:17:57.367 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:17:57.378 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:57.382+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:57.475 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.476 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.479 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:57.483+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:57.485 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:17:57.580 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:57.583+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:57.590 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.591 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.600 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:17:57.681 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:57.684+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:57.707 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.708 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:17:57.730 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":20,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":9,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:17:57.783 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:57.787+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:57.835 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.835 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:57.844 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:17:57.884 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:57.887+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:57.985 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:57.988+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:57.991 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:57.991 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:58.001 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:17:58.085 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.088+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.146 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:58.146 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:58.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:17:58.157 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:17:58.186 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.189+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.262 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:17:58.263 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:17:58.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:17:58.287 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.290+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.322 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:17:58.323 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:17:58.340 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:17:58.340 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:17:58.358 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:17:58.358 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:17:58.375 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:17:58.375 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.104543009, 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:17:58.383 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:17:58.386 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:17:58.387 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:58.388 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:58.391+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:58.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:17:58.393 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:17:58.393 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:17:58.394 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:17:58.411 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:17:58.411 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:17:58.429 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:17:58.429 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:17:58.433 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:17:58.433 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:17:58.489 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:58.492+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:58.500 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:58.501 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:17:58.501 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:58.510 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:17:58.590 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.592+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.657 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:58.657 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:17:58.657 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:17:58.657 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:17:58.657 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:58.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:17:58.691 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.694+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.772 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:17:58.773 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:17:58.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:17:58.773 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:17:58.773 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:17:58.781 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:17:58.826 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:58.829+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:58.893 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:17:58.896+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:17:58.898 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:58.898 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:58.898 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:58.898 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:58.908 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:17:58.994 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:58.998+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:59.054 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:59.055 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:59.055 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.063 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:17:59.096 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:17:59.099+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:17:59.168 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:59.169 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:59.169 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.178 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:17:59.221 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:59.224+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:59.283 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.283 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:59.284 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:59.284 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:17:59.297 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:59.301+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:59.382 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:17:59.426 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:59.430+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:59.439 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.439 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:59.439 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:59.439 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:59.439 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.448 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:17:59.499 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:17:59.503+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:17:59.555 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.555 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:59.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:59.555 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:17:59.555 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.562 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:17:59.600 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:17:59.604+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:17:59.668 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:17:59.668 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.677 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:17:59.721 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:17:59.724+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:17:59.783 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.783 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:17:59.783 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:17:59.784 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:17:59.784 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:17:59.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:17:59.802 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:17:59.806+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:17:59.903 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:17:59.906+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:17:59.941 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:17:59.941 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:17:59.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:17:59.941 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:17:59.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.140653380000003, 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:17:59.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:17:59.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:17:59.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:17:59.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.948 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.107689789000005, 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.949 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:17:59.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:17:59.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:17:59.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.089925374000003, 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:17:59.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:17:59.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:17:59.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.089832344000001, 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:17:59.961 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:17:59.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:17:59.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:17:59.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.061734318999996, 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.968 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:17:59.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:17:59.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.061624241999994, 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.974 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:17:59.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:17:59.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 86.037763713999993, 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.984 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:17:59.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.990 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.896404106000006, 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:17:59.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:17:59.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:17:59.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.896286911000004, 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:17:59.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.896261960000004, 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.004 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:00.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:00.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:00.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.011 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498391101999999, 2024-05-08T17:18:00.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:00.019 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498375522000003, 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.020 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:00.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:00.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:00.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:00.029 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498356201999997, 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:00.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:00.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:00.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:00.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:00.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.036 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498279698999994, 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:00.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:00.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:00.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498258104000001, 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:00.042 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:00.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:00.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498231709999999, 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:00.048 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:00.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:00.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:00.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:00.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.498157484000004, 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.054 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:00.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:00.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.059 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.012363987000001, 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.060 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:00.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:00.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.012350699999999, 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:00.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:00.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:00.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 85.012281631999997, 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:00.072 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:00.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.077 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:00.078 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:00.078 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:00.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:00.080 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:00.083+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:00.183 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:00.186+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:00.189 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:00.189 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:00.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:00.283 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:00.286+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:00.344 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:00.344 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:00.344 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:00.344 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:00.344 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:00.352 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:00.372 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:18:00.384 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:00.387+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:00.435 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:00.463 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:00.463 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.660571876999995, 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:00.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.468 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.638598724999994, 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.472 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.638142368999993, 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:00.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.619100613000001, 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.477 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:00.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:00.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.610811553999994, 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:00.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:00.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.610323550000004, 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.486 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:00.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:00.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:00.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.609849116999996, 2024-05-08T17:18:00.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:00.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.494 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.609455741999994, 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:00.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.603269304999998, 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.499 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:00.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.598465853999997, 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:00.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:00.507 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.595890396000001, 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.593104699999998, 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.512 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.588102077000002, 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:00.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:00.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.416926016000005, 2024-05-08T17:18:00.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:00.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.006342046, 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:00.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 85.533544934999995, 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.530 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:00.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:00.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:00.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 85.532959368999997, 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.534 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:00.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 85.532519961999995, 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:00.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 85.531871976999994, 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:00.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:00.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 85.526410249999998, 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:00.547 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:00.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:00.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:00.551 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:00.553 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:00.553 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:00.556+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:00.557 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:00.557 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:00.577 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:00.577 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:00.594 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:00.594 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:00.611 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:00.611 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:00.629 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:00.629 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:00.629 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:00.647 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:00.647 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:00.654 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:00.657+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:00.664 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:00.664 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:00.666 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:00.666 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:00.668 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:00.668 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:00.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:00.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:00.762 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:00.765+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:00.828 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:00.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:00.883 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:00.886+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:00.948 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:00.948 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:00.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:00.959 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:00.962+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:01.060 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:01.064+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:01.099 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:01.099 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:01.099 INFO:tasks.thrashosds.thrasher:reset_purged_snaps_last 2024-05-08T17:18:01.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 reset_purged_snaps_last 2024-05-08T17:18:01.105 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.105 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.114 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:01.131 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":22,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:01.145 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:18:01.145 INFO:tasks.mon_recovery:stopping all monitors 2024-05-08T17:18:01.146 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T17:18:01.146 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:01.158 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T17:18:01.159 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T17:18:01.159 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:01.200 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T17:18:01.200 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T17:18:01.200 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:01.201 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:01.203+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:01.243 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.243 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.253 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:01.254 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T17:18:01.254 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2024-05-08T17:18:01.254 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T17:18:01.254 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T17:18:01.257 INFO:tasks.ceph.mon.b:Started 2024-05-08T17:18:01.257 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T17:18:01.257 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T17:18:01.262 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:01.265+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:01.297 INFO:tasks.ceph.mon.c:Started 2024-05-08T17:18:01.297 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:18:01.297 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:01.363 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:01.366+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:01.404 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.404 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:01.464 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:01.467+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:01.524 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.525 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.533 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:01.576 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:01.578+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:01.666 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:01.669+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:01.681 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.681 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:01.768 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:01.771+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:01.795 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.795 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.804 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:01.869 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:01.872+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:01.921 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:01.921 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:01.930 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:01.975 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:01.978+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:02.036 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.036 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:02.086 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:02.089+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:02.172 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:02.175+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:02.190 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.190 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.199 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:02.273 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:02.276+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:02.304 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.304 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.314 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:02.375 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:02.377+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:02.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:18:02.460 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.460 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.468 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:02.512 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:02.515+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:02.573 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.574 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.576 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:02.579+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:02.582 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:02.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:02.677 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:02.680+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:02.728 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.728 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.737 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:02.778 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:02.781+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:02.879 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:02.881+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:02.881 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.882 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:02.891 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:02.980 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:02.983+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:02.998 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:02.998 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.007 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:03.081 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:03.084+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:03.116 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.116 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:03.182 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:03.185+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:03.271 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.271 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:03.283 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:03.286+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:03.384 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:03.387+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:03.387 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.387 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.396 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:03.485 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:03.488+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:03.502 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.502 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:03.586 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:03.589+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:03.656 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.656 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:03.708 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:03.711+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:03.770 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.771 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.780 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:03.822 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:03.824+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:03.888 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:03.891+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:03.927 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:03.928 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:03.936 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:04.021 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:04.024+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:04.083 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.083 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.090 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:04.093+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:04.092 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:04.191 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:04.194+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:04.238 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.238 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:04.292 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:04.295+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:04.393 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:04.396+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:04.393 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.394 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:04.494 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:04.497+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:04.508 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.508 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.517 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:04.601 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:04.604+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:04.663 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.663 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.672 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:04.716 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:04.719+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:04.789 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.789 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.797 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:04.800+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:04.798 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:04.898 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:04.901+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:04.945 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:04.945 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:04.954 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:04.999 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:05.002+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:05.059 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.059 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:05.102 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:05.105+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:05.174 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.174 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.184 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:05.203 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:05.206+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:05.289 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.289 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.297 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:05.306 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:05.309+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:05.402 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.402 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.407 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:05.410+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:05.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:05.510 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:05.513+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:05.517 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.517 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.527 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:05.611 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:05.614+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:05.631 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.631 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.640 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:05.715 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:05.717+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:05.745 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.745 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:05.816 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:05.818+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:05.873 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:05.873 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:05.883 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:05.925 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:05.927+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:06.018 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:06.020+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:06.030 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.030 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.039 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:06.120 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:06.123+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:06.145 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.145 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.154 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:06.221 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:06.223+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:06.261 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.261 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:06.322 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:06.325+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.416 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.424 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:06.427+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:06.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:06.525 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:06.527+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:06.530 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.531 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.540 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:06.626 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:06.629+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:06.687 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.687 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:06.727 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:06.730+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:06.803 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.803 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:06.828 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:06.831+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:06.923 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:06.923 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:06.929 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:06.931+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:06.933 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:07.029 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:07.032+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:07.080 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:07.080 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:07.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:07.133 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:07.136+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:07.197 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:07.198 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:07.198 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:07.199 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:07.208 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:07.216 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:07.216 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:07.233 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:07.233 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:07.233 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:07.236+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:07.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 reset_purged_snaps_last 2024-05-08T17:18:07.252 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:07.252 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:07.270 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:07.270 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:07.271 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:07.289 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:07.289 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:07.307 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:07.307 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:07.310 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:07.311 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:07.328 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:07.328 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:07.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:07.328 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:07.328 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:07.332 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:07.335+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:07.338 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:07.433 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:07.435+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:07.456 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:07.456 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:07.456 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:07.466 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:07.534 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:07.536+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:07.558 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:18:07.560+0000 7fe75f7fe640 0 --2- 172.21.15.154:0/4144090541 >> [v2:172.21.15.154:3301/0,v1:172.21.15.154:6790/0] conn(0x7fe76013fa60 0x7fe76014ce90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T17:18:07.571 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:07.571 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:07.571 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:07.571 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:07.571 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:07.580 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:07.597 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 reset_purged_snaps_last 2024-05-08T17:18:07.622 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:18:07.637 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:07.639+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:07.686 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:07.686 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:07.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:07.686 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:07.686 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:07.695 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:07.709 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":24,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:07.722 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:18:07.722 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T17:18:07.723 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T17:18:07.725 INFO:tasks.ceph.mon.a:Started 2024-05-08T17:18:07.725 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:18:07.725 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:07.764 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 reset_purged_snaps_last 2024-05-08T17:18:07.765 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:07.766+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:07.800 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:07.800 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:07.800 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:07.809 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:07.852 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:07.855+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:07.915 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:07.915 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:07.915 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:07.915 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:07.915 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:07.924 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:07.963 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 reset_purged_snaps_last 2024-05-08T17:18:07.965 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:07.968+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:08.038 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:08.040+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:08.070 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:08.070 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:08.070 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:08.079 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:08.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 reset_purged_snaps_last 2024-05-08T17:18:08.117 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":28,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:08.131 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:18:08.131 INFO:tasks.mon_recovery:stopping all monitors 2024-05-08T17:18:08.131 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T17:18:08.131 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:08.139 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:08.142+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:08.143 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T17:18:08.143 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T17:18:08.143 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:08.186 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T17:18:08.187 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T17:18:08.187 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:08.188 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:08.188 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:08.188 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:08.188 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:08.198 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:08.238 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T17:18:08.238 INFO:tasks.mon_recovery:forming a minimal quorum for ['b', 'c', 'a'], then adding monitors 2024-05-08T17:18:08.238 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T17:18:08.239 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T17:18:08.242 INFO:tasks.ceph.mon.b:Started 2024-05-08T17:18:08.242 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T17:18:08.242 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T17:18:08.279 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:08.281+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:08.280 INFO:tasks.ceph.mon.c:Started 2024-05-08T17:18:08.280 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:18:08.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:08.305 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:08.305 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:08.305 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:08.305 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:08.305 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:08.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:08.340 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:08.343+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:08.442 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:08.445+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.665370499000005, 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.467 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.632406907999993, 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:08.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:08.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.614642493000005, 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:08.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:08.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.489 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.614549463000003, 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.490 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:08.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:08.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:08.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:08.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:08.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.586451437999997, 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:08.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:08.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:08.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.505 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.586341360999995, 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.506 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:08.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:08.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:08.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.512 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.562480832999995, 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:08.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:08.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:08.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.519 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.421121224999993, 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:08.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:08.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:08.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:08.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:08.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.421004030000006, 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:08.530 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.531 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:08.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:08.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:08.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:08.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:08.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:08.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:08.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:08.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:08.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:08.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.543 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.543 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.420979079000006, 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.544 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.545 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:08.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:08.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.023108221000001, 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:08.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:08.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:08.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:08.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:08.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.562 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.023092641000005, 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:08.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:08.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:08.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.023073320999998, 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:08.569 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.576 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.022996817999996, 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.577 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:08.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:08.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.022975223000003, 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.590 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.022948829000001, 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.592 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:08.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.592 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:08.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 94.022874603000005, 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:08.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.537081106000002, 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:08.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:08.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:08.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:08.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:08.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:08.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.537067819000001, 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 93.536998750999999, 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:08.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:08.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:08.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.624 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:08.624 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:08.624 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:08.625 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:08.628+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:08.626 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:08.726 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:08.729+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:08.733 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:08.733 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:08.742 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:08.827 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:08.830+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:08.848 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:08.848 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:08.848 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:08.848 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:08.848 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:08.857 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:08.928 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:08.930+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:08.966 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:08.966 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:08.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:08.966 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:08.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.163062288999996, 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:08.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:08.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:08.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:08.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:08.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:08.974 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.141089136999994, 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.140632780999994, 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.979 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:08.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:08.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:08.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.121591025000001, 2024-05-08T17:18:08.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:08.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:08.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.988 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.113301965999995, 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:08.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:08.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:08.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:08.995 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:08.996 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.112813962000004, 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:08.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:08.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:08.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.112339528999996, 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:09.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.111946153999995, 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.006 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:09.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:09.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:09.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.105759716999998, 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:09.010 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.014 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.100956265999997, 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:09.018 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.098380808000002, 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:09.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:09.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.095595111999998, 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:09.023 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:09.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 95.090592489000002, 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:09.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.919416428000005, 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:09.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:09.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.508832458000001, 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:09.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.036035346999995, 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.041 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:09.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:09.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:09.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.045 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.035449780999997, 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:09.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.035010373999995, 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.034362388999995, 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.054 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:09.058 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 94.028900661999998, 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:09.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.062 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.063 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.063 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.063 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.063 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:09.065 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:09.067+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:09.165 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:09.167+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:09.210 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.211 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:09.211 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:09.211 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.220 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:09.306 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:09.309+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:09.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:09.367 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:09.369+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:09.369 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.369 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:09.369 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:09.369 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:09.369 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.379 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:09.468 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:09.471+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:09.526 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.248064896999999, 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.527 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.533 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:09.576 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:09.579+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:09.637 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:18:09.670 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:09.673+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:09.684 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.684 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:09.684 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:09.684 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.695 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:09.771 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:09.774+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:09.803 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.803 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:09.803 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:09.803 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:09.803 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.812 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:09.872 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:09.875+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 21.639613089000001, 2024-05-08T17:18:09.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:09.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:09.922 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:09.927 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:09.973 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:09.975+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:10.034 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:10.034 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:10.034 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:10.034 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:10.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:10.083 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:10.086+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:10.175 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:10.177+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:10.188 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:10.189 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:10.189 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:10.189 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:10.189 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:10.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:10.282 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:10.285+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:10.344 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:10.344 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:10.344 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:10.344 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:10.344 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:10.353 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:10.397 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:10.400+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:10.470 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:10.471 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:10.471 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:10.471 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:10.477 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:10.480+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:10.480 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:10.578 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:10.581+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:10.626 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:10.626 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:10.626 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:10.626 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:10.626 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:10.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:10.719 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:10.722+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:10.780 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:10.783+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:10.780 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:10.780 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:10.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:10.780 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:10.780 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:10.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:10.881 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:10.884+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:10.935 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:10.936 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:10.936 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:10.936 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:10.945 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:10.986 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:10.989+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:11.083 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:11.086+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:11.090 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:11.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:11.091 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:11.091 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:11.091 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:11.099 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:11.183 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:11.186+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:11.204 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:11.205 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:11.205 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:11.205 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:11.205 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:11.213 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:11.284 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:11.287+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:11.360 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:11.360 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:11.360 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:11.369 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:11.413 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:11.416+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:11.475 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:11.475 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:11.475 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:11.475 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:11.475 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:11.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:11.525 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:11.528+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:11.588 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:11.591+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:11.631 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:11.631 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:11.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.830955571999993, 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:11.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:11.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:11.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:11.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:11.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.797991980999996, 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:11.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:11.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:11.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:11.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:11.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:11.643 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.780227565999994, 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:11.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:11.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:11.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:11.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.649 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.780134536000006, 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:11.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:11.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:11.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.752036511, 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.655 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:11.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.751926433999998, 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.728065905999998, 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.666 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:11.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.586706297999996, 2024-05-08T17:18:11.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.586589102999994, 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.678 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:11.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:11.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:11.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.586564151999994, 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188693294000004, 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.690 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:11.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:11.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.695 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188677713999994, 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:11.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:11.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:11.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:11.700 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188658394000001, 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:11.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:11.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188581890999998, 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:11.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188560296000006, 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:11.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:11.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188533902000003, 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:11.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:11.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:11.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:11.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 97.188459675999994, 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:11.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:11.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:11.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 96.702666179000005, 2024-05-08T17:18:11.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 96.702652892000003, 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:11.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:11.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 96.702583824000001, 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:11.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:11.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:11.747 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:11.749 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:11.751+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:11.749 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:11.850 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:11.852+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:11.868 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:11.868 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:11.868 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:11.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:11.961 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:11.964+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:12.024 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.025 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:12.025 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:12.025 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:12.025 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:12.075 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:12.077+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:12.152 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:12.154+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:12.182 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.182 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:12.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.379467950000006, 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:12.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.357494798000005, 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.187 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:12.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.191 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.357038442000004, 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:12.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:12.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.337996685999997, 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.196 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:12.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:12.200 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.329707627000005, 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:12.204 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.329219623, 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:12.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:12.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.208 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.328745190000006, 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.212 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.328351815000005, 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:12.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:12.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:12.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.322165377999994, 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:12.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:12.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:12.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.317361926999993, 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:12.221 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:12.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:12.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.314786468999998, 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.229 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.312000772999994, 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:12.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.233 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.306998149999998, 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:12.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:12.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 98.135822089000001, 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:12.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.725238118999997, 2024-05-08T17:18:12.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:12.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:12.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:12.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.252441008000005, 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.251855441999993, 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.251 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:12.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:12.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.254 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:12.255 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.251416035000005, 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:12.259 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.250768050000005, 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:12.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:12.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.263 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 97.245306322999994, 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:12.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:12.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.268 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:12.270 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:12.272+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:12.370 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:12.373+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:12.416 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.416 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:12.416 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:12.416 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.425 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:12.471 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:12.474+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:12.530 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.530 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:12.530 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:12.530 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:12.530 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.538 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:12.580 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:12.582+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:12.674 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:12.677+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:12.684 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.684 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.406139329999998, 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:12.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:12.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:12.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:12.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:12.691 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.693 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:12.776 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:12.779+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:12.840 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.840 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:12.840 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:12.840 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:12.894 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:12.895+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:12.955 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:12.956 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:12.956 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:12.956 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:12.956 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:12.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:13.006 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:13.008+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:13.079 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:13.081+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:13.111 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:13.111 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.832287655000002, 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:13.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:13.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:13.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:13.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:13.116 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:13.120 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:13.180 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:13.183+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:13.266 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:13.266 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:13.267 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:13.267 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:13.275 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:13.318 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:13.321+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:13.380 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:13.380 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:13.380 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:13.381 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:13.381 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:13.382 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:13.385+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:13.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:13.482 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:13.485+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:13.535 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:13.535 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:13.535 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:13.545 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:13.584 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:13.586+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:13.661 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:13.662 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:13.671 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:13.684 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:13.687+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:13.777 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:13.777 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:13.777 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:13.777 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:13.777 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:13.786 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:13.786 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:13.788+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:13.840 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:13.841 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:13.858 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:13.858 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:13.874 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:13.875 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:13.886 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:13.888+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:13.891 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:13.891 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:13.891 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:13.892 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:13.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:13.892 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:13.892 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:13.900 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:13.908 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:13.909 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:13.909 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:13.927 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:13.927 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:13.943 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:13.944 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:13.946 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:13.947 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:13.988 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:13.991+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.005 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.005 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.013 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:14.088 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:14.091+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:14.118 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.118 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.128 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:14.189 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:14.192+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.234 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:14.237 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:14.237 INFO:tasks.thrashosds.thrasher:Failed to reset_purged_snaps_last, ignoring 2024-05-08T17:18:14.238 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 reset_purged_snaps_last 2024-05-08T17:18:14.273 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.274 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.282 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:14.321 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:14.324+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.387 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.387 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.390 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:14.393+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:14.396 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:14.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 reset_purged_snaps_last 2024-05-08T17:18:14.500 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:14.503+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.541 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.541 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.551 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:14.593 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:18:14.595 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:14.598+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:14.646 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":30,"quorum":[1,2],"quorum_names":["b","c"],"quorum_leader_name":"b","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:14.656 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.656 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.660 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:18:14.660 INFO:tasks.ceph.mon.a:Restarting daemon 2024-05-08T17:18:14.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-05-08T17:18:14.665 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:14.701 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:14.703+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.702 INFO:tasks.ceph.mon.a:Started 2024-05-08T17:18:14.702 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:18:14.702 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:14.794 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:14.797+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:14.813 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.814 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.823 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:14.895 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:14.898+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:14.931 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:14.932 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:14.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:15.024 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:15.027+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:15.075 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:15.089 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:18:15.089 INFO:tasks.mon_recovery:removing mon b 2024-05-08T17:18:15.089 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T17:18:15.089 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:18:15.090 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.090 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:15.101 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:15.102+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:15.101 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T17:18:15.101 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:18:15.102 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:15.198 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:15.201+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:15.251 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.251 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:15.305 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:15.308+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:15.368 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.368 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:15.400 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:15.403+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:15.501 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:15.503+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:15.507 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:15.531 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.531 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:15.624 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:15.627+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:15.687 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.687 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.696 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:15.704 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:15.707+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:15.804 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:15.808+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:15.842 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.842 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:15.906 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:15.908+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:15.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:15.957 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:15.957 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:15.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:16.009 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:16.011+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:16.074 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:16.075 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:16.092 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:16.093 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:16.108 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:16.111+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:16.109 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:16.110 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:16.119 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:16.119 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:16.126 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:16.127 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:16.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:16.145 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:16.145 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:16.146 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:16.164 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:16.164 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:16.194 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:16.195 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:16.198 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:16.198 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:16.239 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:16.242+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:16.246 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:16.246 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:16.246 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:16.246 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:16.246 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:16.255 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:16.309 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:16.312+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:16.402 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:16.402 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:16.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:16.402 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:16.402 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:16.410 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:16.412+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:16.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:16.511 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:16.514+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:16.527 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:16.528 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:16.528 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:16.537 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:16.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:18:16.652 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:16.655+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:16.682 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:16.682 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:16.683 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:16.683 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:16.692 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:16.712 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:16.714+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:16.797 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:16.797 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:16.797 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:16.806 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:16.813 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:16.816+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:16.911 INFO:teuthology.orchestra.run.smithi154.stderr:2024-05-08T17:18:16.914+0000 7f2a3ffff640 0 --2- 172.21.15.154:0/4126855096 >> v2:172.21.15.154:3300/0 conn(0x7f2a400ac1e0 0x7f2a400aae80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2024-05-08T17:18:16.911 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:16.912 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:16.912 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:16.912 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:16.914 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:16.917+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:16.921 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:17.015 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:17.018+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:17.066 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:17.066 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:17.066 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:17.075 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:17.116 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:17.119+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:17.217 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:17.220+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:17.220 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:17.220 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:17.220 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:17.230 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:17.318 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:17.322+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:17.335 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:17.335 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:17.336 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:17.336 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:17.344 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:17.419 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:17.422+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:17.450 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:17.450 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:17.450 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:17.450 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:17.450 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:17.460 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:17.520 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:17.523+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:17.568 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:17.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.767779183, 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.569 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:17.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:17.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.576 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.734815592, 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:17.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:17.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:17.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.717051177, 2024-05-08T17:18:17.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:17.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:17.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:17.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:17.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.716958147, 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.588 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.68886012199999, 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.595 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.596 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:17.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:17.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:17.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:17.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.68875004500001, 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:17.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:17.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:17.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.610 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.66488951700001, 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.611 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:17.616 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.523529909, 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:17.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:17.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:17.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:17.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:17.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.523412714, 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:17.622 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:17.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:17.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.523387763, 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:17.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.125516905, 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:17.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:17.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:17.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:17.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.647 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.125501325, 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.648 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:17.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.12548200499999, 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.655 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:17.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:17.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.12540550200001, 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:17.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:17.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.667 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.125383907, 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:17.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.125357513, 2024-05-08T17:18:17.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:17.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:17.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:17.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:17.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 103.125283287, 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.679 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:17.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:17.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 102.63948979, 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.685 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 102.639476503, 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:17.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:17.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 102.639407435, 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:17.696 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:17.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:17.702 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:17.703 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:17.706+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:17.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:17.805 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:17.807+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:17.814 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:17.815 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:17.815 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:17.815 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:17.823 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:17.907 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:17.910+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:17.969 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:17.970 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:17.970 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:17.970 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:17.970 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:17.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:18.019 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:18.023+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:18.107 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:18.110+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:18.125 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.322392499, 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:18.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:18.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.129 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.300419347, 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:18.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:18.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.133 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.299962991, 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:18.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:18.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:18.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.28092123499999, 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:18.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.272632176, 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.142 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:18.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:18.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:18.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.272144172, 2024-05-08T17:18:18.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:18.150 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.271669739, 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.271276364, 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:18.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.158 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.26508992700001, 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:18.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:18.162 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.260286476, 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.25771101799999, 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:18.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:18.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.25492532200001, 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:18.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:18.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:18.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.249922699, 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:18.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 104.078746638, 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.66816266799999, 2024-05-08T17:18:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.195365557, 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:18.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.194779991, 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:18.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:18.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:18.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.194340584, 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.193692599, 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:18.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 103.18823087200001, 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.207 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.213 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:18.214 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:18.216 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:18.216 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:18.219+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:18.317 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:18.320+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:18.420 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.420 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:18.420 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:18.420 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:18.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:18.421 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:18.422+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:18.520 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:18.523+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:18.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:18.568 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.568 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:18.568 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:18.568 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:18.568 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:18.577 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:18.622 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:18.625+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:18.723 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:18.725+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:18.725 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.725 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.447016093999999, 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:18.726 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:18.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:18.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:18.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:18.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:18.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:18.733 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:18.735 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:18.826 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:18.828+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:18.882 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:18.882 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:18.882 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:18.891 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:18.891 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":4,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:18.946 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:18.949+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:18.996 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:18.996 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:18.996 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:18.996 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:18.996 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:19.005 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:19.031 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:19.033+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:19.132 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:19.134+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.151 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 30.872653162999999, 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:19.155 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:19.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:19.243 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:19.247+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:19.306 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:19.306 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:19.306 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:19.306 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:19.315 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:19.356 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:19.359+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:19.434 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:19.438+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:19.460 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:19.460 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:19.460 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:19.460 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:19.460 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:19.470 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:19.553 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:19.557+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:19.597 INFO:tasks.thrashosds.thrasher:in_osds: [0, 1, 2, 3, 4, 5, 6, 7] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:19.597 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:19.597 INFO:tasks.thrashosds.thrasher:Removing osd 0, in_osds are: [0, 1, 2, 3, 4, 5, 6, 7] 2024-05-08T17:18:19.597 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd out 0 2024-05-08T17:18:19.616 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:19.616 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:19.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:19.616 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:19.616 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:19.625 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:19.670 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:19.673+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:19.737 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:19.740+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:19.742 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:19.742 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:19.743 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:19.752 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:19.838 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:19.842+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:19.897 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:19.897 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:19.897 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:19.897 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:19.897 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:19.906 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:19.940 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:19.943+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:20.010 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.010 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:20.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:20.011 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:20.011 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.019 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:20.063 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:20.066+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:20.125 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:20.125 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:20.125 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.133 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:20.142 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:20.145+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:20.243 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:20.246+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:20.279 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.280 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:20.280 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:20.280 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:20.280 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.288 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:20.372 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:20.375+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:20.434 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:20.434 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.443 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:20.444 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:20.448+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:20.545 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:20.548+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:20.588 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.589 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:20.589 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:20.589 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.597 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:20.681 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:20.684+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:20.743 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.743 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:20.743 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:20.743 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:20.743 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:20.747 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:20.750+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:20.752 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:20.848 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:20.851+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.899 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.098441943, 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:20.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.065478352, 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.906 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:20.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:20.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:20.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:20.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.047713937, 2024-05-08T17:18:20.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:20.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:20.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:20.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:20.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.047620907, 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.918 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.923 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.019522882, 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:20.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 107.019412805, 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.929 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:20.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.995552277, 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:20.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:20.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:20.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.854192669, 2024-05-08T17:18:20.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:20.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:20.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:20.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:20.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.854075474, 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.946 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:20.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:20.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.854050523, 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.952 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:20.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:20.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.957 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.45617966499999, 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.456164085, 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.963 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:20.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:20.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:20.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.456144765, 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:20.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:20.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:20.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.456068262, 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.974 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:20.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:20.979 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.456046667, 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.985 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.45602027299999, 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:20.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:20.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 106.455946047, 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.97015254999999, 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:20.997 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:20.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:20.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:21.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:21.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.97013926299999, 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:21.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:21.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:21.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:21.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 105.97007019500001, 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:21.008 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:21.013 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:21.014 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:21.014 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:21.015 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:21.016 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:21.018+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:21.117 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:21.120+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:21.124 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.124 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:21.125 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:21.125 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:21.218 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:21.222+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:21.279 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.279 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:21.279 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:21.279 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:21.279 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.288 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:21.329 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:21.332+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:21.420 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:21.424+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.63212966899999, 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:21.435 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:21.439 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.61015651699999, 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.444 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.60970016100001, 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:21.448 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.590658405, 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:21.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:21.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.58236934599999, 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:21.453 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:21.457 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.581881342, 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:21.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:21.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:21.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.58140690899999, 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.465 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.58101353399999, 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:21.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.574827097, 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.470 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.570023646, 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:21.474 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:21.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:21.479 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.567448188, 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.564662492, 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.484 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:21.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.559659869, 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.488 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:21.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:21.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:21.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 107.388483808, 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:21.492 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:21.496 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.977899838, 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:21.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:21.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.500 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.50510272699999, 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.504 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.504517161, 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:21.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.50407775399999, 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.509 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:21.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:21.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.50342976899999, 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:21.517 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 106.497968042, 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:21.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.521 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.523 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:21.523 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:21.526+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:21.623 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:21.626+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:21.669 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.670 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:21.670 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:21.670 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.678 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:21.724 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:21.727+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:21.786 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:21.786 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.795 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:21.838 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:21.842+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:21.900 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 33.621470041000002, 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:21.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:21.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:21.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:21.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:21.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:21.906 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:21.906 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:21.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:21.951 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:21.954+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:22.028 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:22.031+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:22.060 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.060 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:22.060 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:22.060 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.069 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:22.129 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:22.133+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:22.174 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.175 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:22.175 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:22.175 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:22.175 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.184 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:22.230 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:22.233+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:22.271 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":34,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":7,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:22.331 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.053274746, 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:22.332 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:22.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:22.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:22.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:22.336 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.336 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:22.335+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:22.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:22.433 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:22.436+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:22.447 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:22.447 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:22.447 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.456 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:22.540 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:22.544+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:22.602 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.602 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:22.602 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:22.602 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:22.602 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.611 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:22.653 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:22.656+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:22.735 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:22.739+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:22.759 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:22.760 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:22.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:22.760 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:22.760 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:22.768 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:22.836 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:22.840+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:22.885 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:22.885 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:22.885 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:22.894 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:22.938 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:22.941+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:23.000 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:23.000 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:23.000 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.008 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:23.051 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:23.053+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:23.139 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:23.143+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:23.155 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:23.156 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:23.156 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.165 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:23.241 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:23.244+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:23.271 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:23.271 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:23.272 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:23.342 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:23.345+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:23.386 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.386 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:23.387 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:23.387 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.395 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:23.442 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:23.445+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:23.500 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:23.500 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.509 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:23.550 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:23.553+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:23.644 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:23.647+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:23.654 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.654 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:23.654 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:23.655 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.664 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:23.746 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:23.748+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:23.769 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.769 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:23.769 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:23.769 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:23.769 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:23.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:23.862 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:23.865+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:23.927 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.126318395, 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.093354804, 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:23.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:23.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:23.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.939 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.075590389, 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.940 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.945 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.075497359, 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.946 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:23.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:23.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:23.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:23.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.047399334, 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:23.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:23.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:23.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.957 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.047289257, 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:23.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:23.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:23.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 110.023428729, 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.963 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:23.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:23.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.882069121, 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:23.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.881951926, 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.974 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:23.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:23.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.881926975, 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:23.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:23.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:23.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.48405611699999, 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:23.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:23.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:23.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:23.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.484040537, 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.991 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:23.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:23.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:23.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:23.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.997 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.48402121700001, 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:23.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:23.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:24.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:24.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:24.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.004 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.483944714, 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.005 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:24.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.483923119, 2024-05-08T17:18:24.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:24.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.48389672499999, 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.016 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:24.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:24.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 109.483822499, 2024-05-08T17:18:24.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.998029002, 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.028 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:24.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:24.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:24.033 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.99801571499999, 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:24.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 108.99794664700001, 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:24.039 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:24.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:24.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:24.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.044 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:24.045 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:24.045 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:24.046 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:24.049+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:24.046 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:24.147 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:24.151+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:24.164 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.164 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:24.164 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:24.164 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.173 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:24.257 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:24.261+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:24.319 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.319 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:24.319 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:24.319 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:24.319 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.328 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:24.369 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:24.372+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:24.450 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:24.454+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:24.475 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.475 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:24.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:24.475 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.672666549, 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:24.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.650693397, 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:24.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:24.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.650237041, 2024-05-08T17:18:24.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.631195285, 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:24.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.622906226, 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:24.497 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.62241822199999, 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:24.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:24.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.621943789, 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:24.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:24.505 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.621550414, 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:24.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:24.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.615363977, 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.510 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:24.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:24.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.610560526, 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:24.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:24.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.607985068, 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:24.519 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.523 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.605199372, 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.527 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.60019674900001, 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:24.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:24.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.532 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.42902068799999, 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:24.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 110.018436718, 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.537 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:24.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:24.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:24.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 109.545639607, 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.541 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:24.545 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 109.545054041, 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:24.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 109.544614634, 2024-05-08T17:18:24.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:24.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:24.554 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 109.543966649, 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:24.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 109.538504922, 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:24.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:24.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:24.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.564 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.566 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:24.566 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:24.569+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:24.667 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:24.670+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:24.712 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.712 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:24.712 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:24.712 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.721 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:24.767 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:24.771+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:24.829 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.829 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:24.829 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:24.829 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:24.829 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.838 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:24.879 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:24.882+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:24.969 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:24.973+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.984 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 36.705778135000003, 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:24.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.991 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:24.992 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:24.992 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:24.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:25.070 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:25.074+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:25.099 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.099 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:25.099 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:25.099 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.108 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:25.191 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:25.195+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:25.253 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.254 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:25.254 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:25.254 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:25.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.262 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:25.273 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:25.276+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:25.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:25.374 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:25.378+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:25.411 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.133062504000002, 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:25.412 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.413 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:25.414 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:25.415 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:25.416 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.420 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:25.475 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:25.479+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:25.566 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.567 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:25.567 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:25.567 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.575 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:25.578 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:25.580+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:25.677 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:25.681+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:25.682 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.683 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:25.683 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:25.683 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:25.683 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.691 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:25.778 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:25.781+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:25.797 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:25.797 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:25.797 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:25.797 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:25.797 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:25.806 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:25.879 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:25.883+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:25.923 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:25.923 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:25.924 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:25.924 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:25.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:25.980 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:25.983+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:26.039 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.039 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:26.039 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:26.039 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:26.039 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:26.081 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:26.085+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:26.183 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:26.186+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:26.193 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.193 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:26.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:26.193 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:26.193 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.202 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:26.283 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:26.287+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:26.306 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.306 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:26.307 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:26.307 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.315 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:26.384 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:26.387+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:26.421 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.421 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:26.421 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:26.421 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:26.421 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.429 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:26.513 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:26.516+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:26.575 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.575 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:26.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:26.575 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:26.575 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.584 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:26.585 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:26.589+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:26.686 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:26.690+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:26.730 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.730 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:26.730 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:26.730 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.739 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:26.787 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:26.791+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:26.844 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.844 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:26.844 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:26.844 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:26.844 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:26.853 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:26.888 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:26.892+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.161122543, 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.962 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:26.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:26.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:26.965 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:26.966 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:26.967 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.12815895200001, 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:26.968 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.969 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.970 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.971 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.972 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.973 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.110394537, 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.974 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.975 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.976 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.977 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.978 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:26.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.110301507, 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.980 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.981 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.982 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.983 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.984 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.082203482, 2024-05-08T17:18:26.985 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.986 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:26.987 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:26.988 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:26.989 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.990 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.08209340499999, 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.991 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.992 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.993 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.994 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.995 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:26.996 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 113.05823287699999, 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:26.997 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:26.998 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:26.999 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:27.000 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:27.001 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.91687326900001, 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.002 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:27.003 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.005 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.005 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.005 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.006 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.007 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.008 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.009 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.91675607400001, 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:27.010 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:27.011 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:27.012 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:27.013 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:27.014 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.91673112300001, 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.015 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:27.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518860265, 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:27.020 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:27.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518844685, 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:27.026 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:27.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:27.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:27.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:27.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518825365, 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.032 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518748862, 2024-05-08T17:18:27.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:27.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:27.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518727267, 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.043 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:27.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:27.048 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518700873, 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.049 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:27.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:27.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.518626647, 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:27.054 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.059 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.03283315, 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.032819863, 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.065 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:27.070 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 112.032750795, 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:27.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:27.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:27.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:27.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.075 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.076 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:27.076 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:27.076 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:27.077 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:27.078 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:27.080+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:27.178 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:27.182+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:27.196 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.196 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:27.196 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:27.196 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.206 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:27.280 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:27.282+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:27.316 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.316 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:27.316 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:27.316 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:27.316 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.325 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:27.380 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:27.384+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.629885116, 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.433 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:27.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.435 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.607911964, 2024-05-08T17:18:27.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:27.441 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.607455608, 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.588413852, 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:27.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:27.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:27.449 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.580124793, 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.57963678900001, 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.454 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:27.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:27.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.579162356, 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.458 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:27.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.578768981, 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.466 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.572582544, 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.470 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.567779093, 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:27.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:27.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:27.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.565203635, 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.479 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.562417939, 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:27.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.557415316, 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.484 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:27.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.488 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 113.38623925500001, 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.492 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.975655285, 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:27.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.502858174, 2024-05-08T17:18:27.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:27.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:27.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:27.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.501 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.502272608, 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:27.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.501833201, 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.501185216, 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 112.495723489, 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:27.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.518 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.520 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:27.520 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:27.523+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:27.620 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:27.624+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:27.666 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.666 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:27.666 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:27.666 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:27.721 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:27.725+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:27.821 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.821 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:27.821 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:27.821 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:27.821 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.822 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:27.826+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:27.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:27.923 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:27.927+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:27.937 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 39.658048884999999, 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:27.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:27.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:27.942 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:27.945 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:28.025 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:28.028+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:28.092 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.093 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:28.093 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:28.093 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:28.126 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:28.129+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:28.208 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.208 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:28.208 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:28.208 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:28.208 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.217 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:28.261 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:28.265+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:28.322 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.044092874999997, 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:28.323 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:28.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:28.327 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:28.328 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.329 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:28.332+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:28.331 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:28.433 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:28.434+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:28.477 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.478 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:28.478 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:28.478 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:28.570 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:28.574+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:28.631 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:28.635+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:28.633 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:28.633 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:28.633 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:28.732 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:28.736+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:28.790 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:28.790 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:28.790 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:28.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:28.790 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:28.799 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:28.843 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:28.846+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:28.916 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:28.916 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:28.916 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:28.916 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:28.926 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:28.934 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:28.938+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:29.035 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:29.038+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:29.072 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.072 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:29.072 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:29.073 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:29.073 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.081 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:29.136 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:29.140+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:29.186 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.187 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:29.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:29.187 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:29.187 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.195 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:29.239 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:29.242+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:29.300 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.301 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:29.301 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:29.301 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.309 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:29.350 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:29.353+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:29.439 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:29.442+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:29.458 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.458 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:29.458 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:29.458 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:29.458 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:29.548 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:29.551+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:29.610 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:29.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:29.610 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:29.663 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:29.666+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:29.741 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:29.744+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:29.768 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.768 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:29.768 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:29.768 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.777 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:29.842 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:29.846+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:29.878 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-05-08T17:18:29.881+0000 7f7b5e0ac640 -1 mon.a@0(leader).osd e31 definitely_dead 0 2024-05-08T17:18:29.897 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:29.897 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:29.916 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:29.916 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:29.922 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:29.922 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:29.922 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:29.922 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:29.922 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:29.931 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:29.933 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:29.933 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:29.942 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:29.946+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:29.950 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:29.950 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:29.969 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:29.969 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:29.988 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:29.988 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:30.004 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:30.005 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:30.008 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:30.008 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:30.039 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.039 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.043 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:30.046+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:30.048 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:30.145 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:30.148+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:30.149 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:30.156 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.156 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.163 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:18:30.163 INFO:tasks.mon_recovery:causing some monitor log activity 2024-05-08T17:18:30.163 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2024-05-08T17:18:30.166 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:30.246 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:30.249+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:30.313 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.313 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.324 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:30.368 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:30.372+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:30.432 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.432 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.441 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:30.447 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:30.451+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:30.548 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:30.552+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:30.589 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.589 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.598 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:30.681 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:30.685+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:30.744 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.744 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.750 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:30.754+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:30.752 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:30.851 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:30.854+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:30.886 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:18:30.886 INFO:teuthology.orchestra.run.smithi154.stderr:marked out osd.0. 2024-05-08T17:18:30.903 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2024-05-08T17:18:30.909 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:30.909 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:30.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:30.959 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:30.963+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:31.053 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:31.057+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:31.070 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.070 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:31.163 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:31.167+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:31.225 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.226 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.235 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:31.255 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:31.258+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:31.356 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:31.360+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:31.381 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.381 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:31.457 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:31.460+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:31.537 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.537 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.546 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:31.558 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:31.562+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:31.651 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.651 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:31.660 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:31.663+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:31.761 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:31.764+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:31.805 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.806 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.815 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:31.861 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:31.864+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:31.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2024-05-08T17:18:31.931 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:31.932 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:31.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:31.985 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:31.988+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:32.009 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:32.046 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:32.046 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:32.055 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:32.063 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:32.066+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:32.136 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:32.136 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:32.157 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:32.164 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:32.167+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:32.174 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:32.174 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:32.191 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:32.191 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:32.201 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:32.201 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:32.209 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:32.209 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:32.209 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:32.210 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:32.227 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:32.227 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:32.244 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:32.245 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:32.248 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:32.248 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:32.265 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:32.268+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:32.316 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:32.316 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:32.316 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:32.326 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:32.366 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:32.370+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:32.467 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:32.470+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:32.471 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:32.471 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:32.480 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:32.568 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:32.572+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:32.626 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:32.627 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:32.627 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:32.635 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:32.679 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:32.682+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:32.740 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:32.740 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:32.740 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:32.740 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:32.749 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:32.790 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:32.793+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:32.871 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:32.875+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:32.895 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:32.895 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:32.895 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:32.904 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:32.905 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:18:32.906 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2024-05-08T17:18:32.992 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:32.996+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.013 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.212505042, 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:33.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:33.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.020 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.17954145100001, 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.021 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.022 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.023 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:33.024 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.025 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.026 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.161777036, 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.027 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:33.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.032 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.033 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.033 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.033 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.161684006, 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.035 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.036 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.037 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.038 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.039 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.133585981, 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.040 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.041 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.042 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.043 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:33.044 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.13347590399999, 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:33.045 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.046 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:33.047 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:33.048 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.049 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 119.10961537599999, 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:33.051 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:33.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.054 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.96825576800001, 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.057 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.058 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.059 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.060 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:33.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.968138573, 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:33.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.968113622, 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:33.068 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:33.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570242764, 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.074 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.075 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:33.076 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:33.077 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:33.078 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.079 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570227184, 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.080 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:33.081 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.082 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.083 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.084 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.085 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570207864, 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:33.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.57013136099999, 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.092 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:33.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.097 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570109766, 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:33.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:33.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:33.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:33.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570083372, 2024-05-08T17:18:33.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:33.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:33.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:33.108 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.570009146, 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.109 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:33.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:33.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:33.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.114 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.084215649, 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.084202362, 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.120 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:33.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:33.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:33.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 118.084133294, 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:33.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:33.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:33.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:33.131 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:33.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:33.133 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:33.136+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:33.233 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:33.236+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:33.240 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.240 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:33.240 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:33.240 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:33.335 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:33.338+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:33.356 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.356 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:33.356 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:33.356 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:33.356 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.364 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:33.436 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:33.439+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:33.471 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.668294549, 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.475 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.64632139699999, 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:33.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:33.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.64586504099999, 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:33.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.626823285, 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:33.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:33.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:33.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.61853422599999, 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.489 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.618046222, 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.495 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:33.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:33.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.617571789, 2024-05-08T17:18:33.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:33.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.61717841399999, 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.610991977, 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:33.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.606188526, 2024-05-08T17:18:33.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.518 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.603613068, 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.520 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.522 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.600827372, 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:33.523 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:33.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:33.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.595824749, 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.527 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:33.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.424648688, 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 119.014064718, 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:33.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:33.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:33.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:33.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 118.54126760699999, 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 118.540682041, 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.544 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:33.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 118.54024263399999, 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.548 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:33.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 118.53959464899999, 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:33.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:33.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 118.534132922, 2024-05-08T17:18:33.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.560 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.562 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:33.562 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:33.565+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:33.663 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:33.666+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:33.667 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.667 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:33.667 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:33.668 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.676 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:33.764 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:33.767+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:33.782 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.782 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:33.782 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:33.782 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:33.783 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:33.864 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:33.868+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:33.886 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:18:33.900 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 45.621937131000003, 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:33.901 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:33.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:33.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.907 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:33.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:33.907 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:33.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2024-05-08T17:18:33.910 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:33.997 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:34.000+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:34.059 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:34.059 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:34.059 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:34.067 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:34.070+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:34.070 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:34.167 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:34.171+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:34.219 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:34.219 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:34.219 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:34.219 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:34.219 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:34.229 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:34.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:34.270 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:34.273+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:34.369 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:34.373+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.378 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 46.098449957, 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:34.379 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:34.380 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:34.381 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:34.382 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:34.387 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:34.392 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:34.392 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:34.410 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:34.410 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:34.428 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:34.428 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:34.445 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:34.445 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:34.463 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:34.463 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:34.463 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:34.470 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:34.474+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:34.481 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:34.481 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:34.499 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:34.499 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:34.499 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:34.500 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:34.502 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:34.503 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:34.509 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:34.572 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:34.575+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:34.656 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:34.656 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:34.666 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:34.709 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:34.713+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:34.773 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:34.773 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:34.773 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:34.777+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:34.782 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:34.874 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:34.878+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:34.899 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:34.899 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:34.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2024-05-08T17:18:34.909 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:34.994 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:34.997+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:35.055 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.055 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:35.076 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:35.079+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:35.177 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:35.180+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:35.210 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.211 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:35.278 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:35.282+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:35.325 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.325 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.334 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:35.378 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:35.381+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:35.478 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:35.482+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:35.480 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.480 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.489 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:35.579 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:35.583+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:35.593 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.593 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.601 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:35.680 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:35.684+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:35.707 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.707 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.717 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:35.781 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:35.785+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:35.863 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.863 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.873 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:35.882 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:35.886+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:35.902 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:35.902 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:35.902 INFO:tasks.thrashosds.thrasher:inject_pause on osd.7 2024-05-08T17:18:35.902 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:18:35.903 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:18:35.903 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok config set filestore_inject_stall 3 2024-05-08T17:18:35.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:18:35.913 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2024-05-08T17:18:35.980 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:35.981 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:35.983 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:35.986+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:35.991 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:36.022 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:36.023 INFO:teuthology.orchestra.run.smithi183.stdout: "success": "" 2024-05-08T17:18:36.023 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:36.084 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:36.087+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:36.134 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:36.134 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:36.143 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:36.185 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:36.188+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:36.251 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:36.251 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:36.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:36.285 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:36.288+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:36.365 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:36.365 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:36.374 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:36.417 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:36.421+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:36.480 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:36.480 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:36.487 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:36.490+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:36.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:36.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:36.587 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:36.590+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:36.642 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:36.642 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:36.655 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:36.663 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:36.663 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:36.681 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:36.681 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:36.688 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:36.692+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:36.698 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:36.698 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:36.715 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:36.716 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:36.734 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:36.734 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:36.734 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:36.752 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:36.752 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 48.485247897000001, 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:36.764 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:36.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:36.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:36.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:36.769 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:36.770 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:36.773 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:36.774 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:36.774 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:36.790 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:36.792+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:36.890 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:36.894+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:36.906 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:18:36.915 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2024-05-08T17:18:36.924 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:36.924 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:36.924 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:36.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:37.017 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:37.021+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:37.082 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:37.082 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:37.082 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:37.083 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:37.083 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:37.092 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:37.093 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:37.095+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:37.192 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:37.195+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.242 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 48.961746411, 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:37.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:37.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:37.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:37.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:37.247 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:37.249 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:37.294 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:37.297+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:37.354 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:37.354 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:37.354 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:37.354 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:37.363 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:37.406 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:37.410+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:37.469 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:37.469 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:37.469 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:37.469 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:37.469 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:37.478 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:37.519 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:37.522+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:37.596 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:37.600+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:37.623 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:37.624 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:37.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:37.624 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:37.624 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:37.633 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:37.697 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:37.701+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:37.750 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:37.751 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:37.751 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:37.751 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:37.760 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:37.798 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:37.801+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:37.866 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:37.866 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:37.866 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:37.874 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:37.898 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:37.902+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:37.915 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2024-05-08T17:18:37.979 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:37.979 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:37.979 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:37.980 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:37.980 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:37.988 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:38.032 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:38.035+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:38.094 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.094 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:38.095 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:38.095 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.101 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:38.104+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:38.103 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:38.202 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:38.206+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:38.249 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.249 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:38.249 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:38.249 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:38.249 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.258 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:38.303 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:38.307+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1720333199999997, 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.364 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:18:38.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.367 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 4.1689978019999998, 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:38.368 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:18:38.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:18:38.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.372 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:38.405 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:38.409+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:38.478 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:38.479 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:38.479 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.488 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:38.505 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:38.509+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:38.593 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:38.594 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:38.594 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.602 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:38.606 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:38.610+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:38.707 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:38.710+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:38.709 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:38.709 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:38.710 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.90823851899999, 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:38.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:38.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.875274928, 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.717 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:38.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:38.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:38.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.722 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.85751051299999, 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.723 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:38.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.728 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.85741748300001, 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.729 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.730 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:38.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:38.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:38.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.829319458, 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.735 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.736 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:38.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.741 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.829209381, 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.742 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:38.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:38.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.805348853, 2024-05-08T17:18:38.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:38.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.663989245, 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.753 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:38.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:38.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:38.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.66387204999999, 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.759 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:38.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:38.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:38.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:38.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.766 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:38.767 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.66384709899999, 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.768 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:38.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:38.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.265976241, 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.777 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:38.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:38.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:38.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.783 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.26596066099999, 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.784 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:38.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:38.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.792 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.265941341, 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:38.793 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:38.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:38.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:38.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:38.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:38.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:38.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.265864838, 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.805 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:38.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:38.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:38.811 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.26584324300001, 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.812 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:38.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:38.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:38.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.265816849, 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:38.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:38.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:38.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.825 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 124.26574262299999, 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.826 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:38.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:38.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:38.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 123.77994912600001, 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.831 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:38.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:38.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:38.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 123.779935839, 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:38.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.841 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:38.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.843 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 123.779866771, 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.844 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:38.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:38.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:38.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:38.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:38.849 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:38.849 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:38.851 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:38.852 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:38.854+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:38.922 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2024-05-08T17:18:38.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:18:38.963 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:38.966+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:38.966 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:38.966 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:38.966 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:38.966 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:38.976 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:38.982 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:38.982 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:39.002 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:39.002 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:39.020 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:39.021 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:39.038 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:39.039 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:39.052 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:39.055+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:39.056 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:39.056 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:39.057 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:39.079 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:39.079 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:39.096 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.096 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:39.111 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:39.111 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:39.116 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:39.117 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:39.158 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:39.162+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:39.218 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.218 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.227 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:39.268 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:39.272+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:39.356 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:39.358+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:39.374 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.374 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.383 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:39.457 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:39.460+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:39.489 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.489 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.498 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:39.558 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:39.561+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:39.604 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.604 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:39.659 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:39.661+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:39.720 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.720 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.729 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:39.769 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:39.772+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:39.861 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:39.863+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:39.878 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:39.878 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:39.884 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:39.911 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:18:39.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2024-05-08T17:18:39.963 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:39.965+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:40.021 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.021 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.031 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:40.064 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:40.067+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:40.164 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:40.167+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:40.176 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.176 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.186 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:40.266 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:40.268+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:40.293 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.293 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.302 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:40.367 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:40.369+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:40.409 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.409 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.418 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:40.468 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:40.471+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:40.536 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.536 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.546 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:40.569 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:40.572+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:40.651 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.651 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.660 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:40.704 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:40.707+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:40.766 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.766 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.770 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:40.773+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:40.776 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:40.872 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:40.875+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:40.921 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:40.921 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:40.931 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:40.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2024-05-08T17:18:40.975 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:40.978+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:41.032 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7] out_osds: [0] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:41.033 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:41.033 INFO:tasks.thrashosds.thrasher:Adding osd 0 2024-05-08T17:18:41.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd in 0 2024-05-08T17:18:41.036 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:41.036 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:41.045 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:41.086 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:41.089+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:41.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:41.174 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:41.177+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:41.191 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:41.191 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:41.200 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:41.246 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:41.246 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:41.263 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:41.264 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:41.280 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:41.280 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:41.284 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:41.287+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:41.297 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:41.297 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:41.314 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:41.315 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:41.315 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:41.333 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:41.333 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:41.346 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:41.347 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:41.347 INFO:tasks.ceph.mon.a.smithi154.stderr:2024-05-08T17:18:41.349+0000 7f7b5e0ac640 -1 mon.a@0(leader).osd e38 definitely_dead 0 2024-05-08T17:18:41.349 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:41.350 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:41.353 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:41.353 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:41.356 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:41.375 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:41.378+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:41.477 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:41.480+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:41.501 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:41.502 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:41.510 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:41.594 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:41.597+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.658 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.857423036, 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.824459445, 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:41.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:41.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:41.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:41.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.80669503, 2024-05-08T17:18:41.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:41.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:41.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.806602, 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.676 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:41.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:41.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:41.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:41.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.77850397500001, 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:41.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:41.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.687 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.778393898, 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:41.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:41.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:41.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.75453337, 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.694 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:41.700 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.613173762, 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:41.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:41.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.613056567, 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:41.706 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:41.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:41.711 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.613031616, 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.712 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.713 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.714 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:41.715 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:41.716 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:41.717 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.718 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.215160758, 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.719 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.720 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.721 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.722 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:41.723 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.215145178, 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:41.724 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.725 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.726 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.727 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.728 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.729 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.21512585799999, 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:41.730 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:41.731 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:41.732 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.733 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.734 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.735 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.215049355, 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.21502776, 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.741 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.743 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.747 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.215001366, 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.748 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:41.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:41.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:41.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:41.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 127.21492714, 2024-05-08T17:18:41.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:41.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:41.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:41.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 126.729133643, 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.759 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:41.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.764 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 126.729120356, 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.765 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:41.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:41.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:41.770 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 126.72905128799999, 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:41.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:41.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:41.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:41.776 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:41.778 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:41.778 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:41.780+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:41.878 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:41.881+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:41.884 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:41.884 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:41.884 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:41.884 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:41.893 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:41.918 INFO:teuthology.orchestra.run.smithi154.stderr:marked in osd.0. 2024-05-08T17:18:41.929 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:18:41.935 INFO:tasks.thrashosds.thrasher:Added osd 0 2024-05-08T17:18:41.941 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2024-05-08T17:18:41.982 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:41.985+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:42.007 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.007 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:42.007 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:42.007 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:42.007 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.017 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:42.100 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:42.103+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:42.168 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.168 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:42.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:42.168 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:42.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.364694542, 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:42.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.34272139000001, 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.34226503400001, 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.177 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:42.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:42.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:42.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.323223278, 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.181 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.185 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.31493421900001, 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:42.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:42.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:42.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.314446215, 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.190 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.31397178200001, 2024-05-08T17:18:42.194 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.195 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:42.196 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:42.197 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.198 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.31357840699999, 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.199 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:42.200 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:42.201 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.202 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.30739197, 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.203 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:42.204 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:42.205 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:42.206 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.30258851900001, 2024-05-08T17:18:42.207 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.208 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:42.209 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:42.210 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.211 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.30001306099999, 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.212 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.213 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:42.214 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:42.215 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.297227365, 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.216 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.217 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:42.218 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:42.219 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.292224742, 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.220 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.221 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.222 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.223 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:42.224 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 128.12104868099999, 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.225 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:42.226 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:42.227 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.228 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.710464711, 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.229 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.230 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.231 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.232 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.23766759999999, 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.233 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:42.234 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:42.235 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:42.236 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.237082034, 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.237 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.238 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.239 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.240 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.23664262699999, 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.242 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.243 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.244 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:42.245 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.23599464199999, 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:42.247 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:42.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.249 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.230532915, 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.250 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.251 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:42.252 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.253 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.254 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.255 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:42.256 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:42.257+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:42.355 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:42.358+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:42.402 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.402 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:42.402 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:42.402 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.410 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:42.494 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:42.497+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:42.556 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.556 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:42.557 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:42.557 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.557 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:42.559+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:42.567 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:42.657 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:42.660+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:42.713 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.713 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:42.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.434884211000004, 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:42.714 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:42.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:42.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:42.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:42.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.719 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:42.719 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:42.719 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.722 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:42.764 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:42.766+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:42.858 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:42.861+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:42.870 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:42.870 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:42.870 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:42.870 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:42.879 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:42.919 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:18:42.949 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2024-05-08T17:18:42.959 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:42.962+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:43.030 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:43.031 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:43.031 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:43.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:43.060 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:43.062+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:43.161 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:43.164+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:43.188 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:43.188 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:43.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:43.188 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 54.909877797, 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.189 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:43.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:43.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:43.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:43.193 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:43.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:43.262 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:43.265+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:43.305 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:43.305 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:43.305 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:43.305 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:43.314 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:43.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:43.363 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:43.366+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:43.423 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:43.423 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:43.423 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:43.432 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:43.464 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:43.467+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:43.481 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:43.482 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:43.500 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:43.500 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:43.518 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:43.518 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:43.535 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:43.535 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:43.543 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:43.543 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:43.553 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:43.553 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:43.554 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:43.554 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:43.565 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:43.567+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:43.570 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:43.570 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:43.588 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:43.588 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:43.592 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:43.592 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:43.666 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:43.669+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:43.670 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:43.670 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:43.679 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:43.768 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:43.770+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:43.785 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:43.785 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:43.793 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:43.869 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:43.872+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:43.906 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:43.906 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:43.911 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:43.955 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2024-05-08T17:18:43.995 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:43.998+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:44.056 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.056 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.065 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:44.071 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:44.073+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:44.172 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:44.175+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:44.210 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.211 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.219 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:44.303 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:44.306+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:44.364 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.364 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.374 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:44.374 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:44.376+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:44.474 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:44.477+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:44.519 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.520 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.528 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:44.575 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:44.577+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:44.672 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.673 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.676 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:44.679+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:44.682 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:44.777 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:44.779+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:44.787 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.787 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.796 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:44.878 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:44.880+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:44.904 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:44.904 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:44.912 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:44.937 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:18:44.958 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2024-05-08T17:18:44.978 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:44.981+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:45.022 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.022 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:45.117 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:45.120+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:45.181 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:45.183+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:45.182 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.182 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.192 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:45.281 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:45.284+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:45.338 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.339 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.348 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:45.382 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:45.385+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:45.454 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.455 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:45.507 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:45.510+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:45.569 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.569 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.578 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:45.593 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:45.619 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:45.621+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:45.684 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:45.687+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:45.728 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:45.728 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:45.738 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:45.744 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:45.745 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:45.763 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:45.763 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:45.780 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:45.780 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:45.785 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:45.788+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:45.798 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:45.799 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:18:45.817 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:45.818 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:45.818 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:45.835 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:45.836 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:45.853 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:45.853 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:45.856 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:45.856 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:45.857 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:45.857 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:45.857 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:45.857 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:45.858 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:45.866 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:45.887 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:45.890+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:45.959 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2024-05-08T17:18:45.988 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:45.991+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:46.015 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 57.736514984000003, 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:46.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:46.019 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:46.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:46.089 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:46.092+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:46.137 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:46.137 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:46.138 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:46.138 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:46.147 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:46.190 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:46.193+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:46.254 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:46.254 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:46.254 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:46.254 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:46.263 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:46.306 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:46.309+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:46.368 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:46.369 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:46.369 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:46.369 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:46.369 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:46.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:46.419 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:46.421+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:46.493 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:46.496+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:46.494 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:46.494 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:46.495 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:46.495 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:46.504 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:46.594 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:46.597+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:46.610 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:46.610 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:46.611 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:46.619 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:46.695 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:46.698+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:46.764 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:46.764 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:46.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:46.764 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:46.764 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:46.773 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:46.817 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:46.820+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:46.878 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:46.878 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:46.878 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:46.879 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:46.887 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:46.897 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:46.901+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:46.936 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:46.936 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:46.936 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T17:18:46.937 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T17:18:46.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2024-05-08T17:18:47.007 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:47.010+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:47.032 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:47.032 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:47.032 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:47.032 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:47.032 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:47.041 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:47.100 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:47.102+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:47.147 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:47.147 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:47.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.955824849000001, 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:47.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:18:47.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:18:47.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.151 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 12.952789331, 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.152 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:18:47.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.156 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:47.157 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:47.202 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:47.205+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 43, 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:16:22.335952+0000", 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:18:45.796895+0000", 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "last_up_change": "2024-05-08T17:17:49.336612+0000", 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "last_in_change": "2024-05-08T17:18:41.921552+0000", 2024-05-08T17:18:47.239 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_num": 5801984, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_set": [ 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "noscrub", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "pglog_hardlimit", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "purged_snapdirs", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_deletes", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "sortbitwise" 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_version": 4, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_snapshot": "", 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_max": 2, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "max_osd": 8, 2024-05-08T17:18:47.240 INFO:teuthology.orchestra.run.smithi154.stdout: "require_min_compat_client": "luminous", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "min_compat_client": "jewel", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "require_osd_release": "squid", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "allow_crimson": false, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "pools": [ 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 1, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": ".mgr", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:32.734319+0000", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:18:47.241 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 1, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 1, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 1, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 1, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 1, 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:18:47.242 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "16", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:18:47.243 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:18:47.244 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:18:47.245 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "options": { 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_max": 32, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_min": 1 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": {} 2024-05-08T17:18:47.246 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 6.9000000953674316, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 6.9000000953674316, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2, 2024-05-08T17:18:47.247 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.248 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 2, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": "rbd", 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:33.268021+0000", 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:18:47.249 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 8, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 8, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 8, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 8, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 8, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.250 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "17", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:18:47.251 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:18:47.252 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "options": {}, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd": {} 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:18:47.253 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 2.619999885559082, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 2.619999885559082, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.86000001430511475, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2.25, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2.25, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:18:47.254 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858", 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.255 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6808", 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6809", 2024-05-08T17:18:47.256 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6811", 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.257 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6810", 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6808/3590430155", 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6809/3590430155", 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6811/3590430155", 2024-05-08T17:18:47.258 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6810/3590430155", 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "7e66ef3b-5581-4f8c-9807-e16ea4ec972e", 2024-05-08T17:18:47.259 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 39, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.260 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6812", 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.261 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6813", 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.262 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6815", 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.263 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6814", 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.264 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6812/2114037860", 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6813/2114037860", 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6815/2114037860", 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6814/2114037860", 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "86378b92-f863-41ec-a3a5-f1dc78645250", 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 24, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.266 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6800", 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:47.267 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.268 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.268 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.268 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.268 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.268 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6801", 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.270 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.270 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.270 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.270 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6803", 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.271 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6802", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6800/2790378686", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6801/2790378686", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6803/2790378686", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6802/2790378686", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.272 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "2fa93019-16da-44c6-ac77-0ef554e24e10", 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.273 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6804", 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:47.274 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6805", 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.275 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6807", 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6806", 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6804/3602122863", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6805/3602122863", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6807/3602122863", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6806/3602122863", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "08750d7a-1ca6-4d68-a62e-893e8d979921", 2024-05-08T17:18:47.277 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6812", 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6813", 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.279 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6815", 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6814", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6812/1597208930", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6813/1597208930", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1597208930", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6814/1597208930", 2024-05-08T17:18:47.281 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "770b03a7-7f15-4427-81ce-e512b68bbb5c", 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 0, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 0, 2024-05-08T17:18:47.282 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 24, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.286 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6801", 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6803", 2024-05-08T17:18:47.287 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:47.288 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.288 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.288 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.288 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6807", 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.289 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6805", 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.290 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6801/3448847701", 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6803/3448847701", 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6807/3448847701", 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6805/3448847701", 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "exists" 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.291 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "ac82b5c6-7db8-4383-b0ac-0a203a4d84cd", 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.292 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.293 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6800", 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6802", 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.295 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6806", 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6804", 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.297 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6800/3679914992", 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6802/3679914992", 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6806/3679914992", 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6804/3679914992", 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.298 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "813f2f3d-68eb-487b-89a1-8b9a5a103485", 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:47.299 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 42, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:47.300 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6808", 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:47.301 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.302 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6809", 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6811", 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.303 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6810", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6808/2695818974", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6809/2695818974", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6811/2695818974", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6810/2695818974", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:47.304 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_xinfo": [ 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.817172+0000", 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.851798+0000", 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:18:47.306 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.765787+0000", 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.790359+0000", 2024-05-08T17:18:47.307 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.824178+0000", 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.308 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.696786+0000", 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 24 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.761200+0000", 2024-05-08T17:18:47.309 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.764552+0000", 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap": [], 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_items": [], 2024-05-08T17:18:47.310 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_primaries": [], 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_temp": [], 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_temp": [], 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklist": {}, 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "range_blocklist": {}, 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profiles": { 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "default": { 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "crush-failure-domain": "osd", 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "k": "2", 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "m": "1", 2024-05-08T17:18:47.311 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin": "jerasure", 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "technique": "reed_sol_van" 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps_queue": [], 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "new_removed_snaps": [], 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "new_purged_snaps": [], 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_node_flags": {}, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "device_class_flags": {}, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": { 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_enabled": false, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_bucket_count": 0, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "degraded_stretch_mode": 0, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "recovering_stretch_mode": 0, 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_bucket": 0 2024-05-08T17:18:47.312 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:47.313 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:47.315 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:47.315 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:47.315 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:47.315 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:47.316 INFO:tasks.thrashosds.thrasher:j is {'epoch': 43, 'fsid': 'c69ba297-505d-4f4e-b390-bd22ef533cfc', 'created': '2024-05-08T17:16:22.335952+0000', 'modified': '2024-05-08T17:18:45.796895+0000', 'last_up_change': '2024-05-08T17:17:49.336612+0000', 'last_in_change': '2024-05-08T17:18:41.921552+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'jewel', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T17:16:32.734319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T17:16:33.268021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 3590430155}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6809', 'nonce': 3590430155}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6811', 'nonce': 3590430155}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 3590430155}]}, 'public_addr': '172.21.15.154:6808/3590430155', 'cluster_addr': '172.21.15.154:6809/3590430155', 'heartbeat_back_addr': '172.21.15.154:6811/3590430155', 'heartbeat_front_addr': '172.21.15.154:6810/3590430155', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7e66ef3b-5581-4f8c-9807-e16ea4ec972e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 2114037860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6813', 'nonce': 2114037860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6815', 'nonce': 2114037860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 2114037860}]}, 'public_addr': '172.21.15.154:6812/2114037860', 'cluster_addr': '172.21.15.154:6813/2114037860', 'heartbeat_back_addr': '172.21.15.154:6815/2114037860', 'heartbeat_front_addr': '172.21.15.154:6814/2114037860', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '86378b92-f863-41ec-a3a5-f1dc78645250', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 24, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 2790378686}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6801', 'nonce': 2790378686}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6803', 'nonce': 2790378686}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 2790378686}]}, 'public_addr': '172.21.15.154:6800/2790378686', 'cluster_addr': '172.21.15.154:6801/2790378686', 'heartbeat_back_addr': '172.21.15.154:6803/2790378686', 'heartbeat_front_addr': '172.21.15.154:6802/2790378686', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2fa93019-16da-44c6-ac77-0ef554e24e10', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 3602122863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6805', 'nonce': 3602122863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6807', 'nonce': 3602122863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 3602122863}]}, 'public_addr': '172.21.15.154:6804/3602122863', 'cluster_addr': '172.21.15.154:6805/3602122863', 'heartbeat_back_addr': '172.21.15.154:6807/3602122863', 'heartbeat_front_addr': '172.21.15.154:6806/3602122863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '08750d7a-1ca6-4d68-a62e-893e8d979921', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1597208930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6813', 'nonce': 1597208930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6815', 'nonce': 1597208930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1597208930}]}, 'public_addr': '172.21.15.183:6812/1597208930', 'cluster_addr': '172.21.15.183:6813/1597208930', 'heartbeat_back_addr': '172.21.15.183:6815/1597208930', 'heartbeat_front_addr': '172.21.15.183:6814/1597208930', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '770b03a7-7f15-4427-81ce-e512b68bbb5c', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6801', 'nonce': 3448847701}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6803', 'nonce': 3448847701}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6807', 'nonce': 3448847701}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6805', 'nonce': 3448847701}]}, 'public_addr': '172.21.15.183:6801/3448847701', 'cluster_addr': '172.21.15.183:6803/3448847701', 'heartbeat_back_addr': '172.21.15.183:6807/3448847701', 'heartbeat_front_addr': '172.21.15.183:6805/3448847701', 'state': ['exists']}, {'osd': 6, 'uuid': 'ac82b5c6-7db8-4383-b0ac-0a203a4d84cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 3679914992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 3679914992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 3679914992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 3679914992}]}, 'public_addr': '172.21.15.183:6800/3679914992', 'cluster_addr': '172.21.15.183:6802/3679914992', 'heartbeat_back_addr': '172.21.15.183:6806/3679914992', 'heartbeat_front_addr': '172.21.15.183:6804/3679914992', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '813f2f3d-68eb-487b-89a1-8b9a5a103485', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 2695818974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6809', 'nonce': 2695818974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6811', 'nonce': 2695818974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 2695818974}]}, 'public_addr': '172.21.15.183:6808/2695818974', 'cluster_addr': '172.21.15.183:6809/2695818974', 'heartbeat_back_addr': '172.21.15.183:6811/2695818974', 'heartbeat_front_addr': '172.21.15.183:6810/2695818974', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.817172+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.851798+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.765787+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.790359+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.824178+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.696786+0000', 'dead_epoch': 24}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.761200+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.764552+0000', 'dead_epoch': 0}], 'pg_upmap': [], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T17:18:47.316 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:18:47.316 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:47.316+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:47.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:47.415 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:47.417+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:47.461 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:47.461 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:47.461 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:47.461 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:47.461 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:47.469 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:47.516 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:47.519+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:47.575 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:47.575 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:47.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:47.575 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.774501028, 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:47.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:47.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:47.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:47.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:47.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:47.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:47.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.74153743700001, 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:47.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:47.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:47.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:47.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.72377302199999, 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.593 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:47.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:47.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.600 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.723679992, 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.601 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:47.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.69558196700001, 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:47.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.69547188999999, 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:47.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.67161136199999, 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.619 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:47.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:47.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.53025175400001, 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.625 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:47.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:47.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:47.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.530134559, 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:47.631 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:47.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:47.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.530109608, 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:47.637 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:47.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:47.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:47.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.13223875, 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.643 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:47.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:47.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:47.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.649 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.13222317, 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:47.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:47.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:47.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.13220385, 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:47.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:47.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:47.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.13212734699999, 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.663 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:47.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:47.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:47.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.132105752, 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:47.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:47.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:47.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.132079358, 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.675 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:47.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 133.13200513199999, 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.681 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:47.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:47.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:47.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:47.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 132.64621163499999, 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:47.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:47.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:47.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:47.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:47.698 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 132.64619834800001, 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.699 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:47.700 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.701 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.702 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:47.703 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:47.704 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 132.64612928, 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:47.705 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:47.706 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:47.707 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.708 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.709 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:47.710 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:47.711 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:47.712 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:18:47.713 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:18:47.714 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:47.715 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":103,"stamp":"2024-05-08T17:18:46.726059+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":219700,"kb_used_data":3524,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754755020,"statfs":{"total":773094113280,"available":772869140480,"internally_reserved":0,"allocated":3608576,"data_stored":2115257,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000801"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":55,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-08T17:18:41.926936+0000","last_change":"2024-05-08T17:16:34.852076+0000","last_active":"2024-05-08T17:18:41.926936+0000","last_peered":"2024-05-08T17:18:41.926936+0000","last_clean":"2024-05-08T17:18:41.926936+0000","last_became_active":"2024-05-08T17:16:34.851798+0000","last_became_peered":"2024-05-08T17:16:34.851798+0000","last_unstale":"2024-05-08T17:18:41.926936+0000","last_undegraded":"2024-05-08T17:18:41.926936+0000","last_fullsized":"2024-05-08T17:18:41.926936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:07:38.786655+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":59,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T17:18:42.930299+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:18:42.930299+0000","last_peered":"2024-05-08T17:18:42.930299+0000","last_clean":"2024-05-08T17:18:42.930299+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:18:42.930299+0000","last_undegraded":"2024-05-08T17:18:42.930299+0000","last_fullsized":"2024-05-08T17:18:42.930299+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:38:45.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":60,"reported_epoch":39,"state":"peering","last_fresh":"2024-05-08T17:18:41.925793+0000","last_change":"2024-05-08T17:18:41.925793+0000","last_active":"2024-05-08T17:18:41.925793+0000","last_peered":"2024-05-08T17:18:41.254005+0000","last_clean":"2024-05-08T17:18:41.254005+0000","last_became_active":"2024-05-08T17:18:32.927869+0000","last_became_peered":"2024-05-08T17:18:32.927869+0000","last_unstale":"2024-05-08T17:18:41.925793+0000","last_undegraded":"2024-05-08T17:18:41.925793+0000","last_fullsized":"2024-05-08T17:18:41.925793+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":69,"reported_epoch":40,"state":"active+clean","last_fresh":"2024-05-08T17:18:42.935613+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:18:42.935613+0000","last_peered":"2024-05-08T17:18:42.935613+0000","last_clean":"2024-05-08T17:18:42.935613+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:18:42.935613+0000","last_undegraded":"2024-05-08T17:18:42.935613+0000","last_fullsized":"2024-05-08T17:18:42.935613+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":54,"reported_epoch":40,"state":"active+undersized","last_fresh":"2024-05-08T17:18:42.930062+0000","last_change":"2024-05-08T17:17:50.342570+0000","last_active":"2024-05-08T17:18:42.930062+0000","last_peered":"2024-05-08T17:18:42.930062+0000","last_clean":"2024-05-08T17:17:48.339260+0000","last_became_active":"2024-05-08T17:17:50.342570+0000","last_became_peered":"2024-05-08T17:17:50.342570+0000","last_unstale":"2024-05-08T17:18:42.930062+0000","last_undegraded":"2024-05-08T17:18:42.930062+0000","last_fullsized":"2024-05-08T17:17:50.341965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:06.433312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":57,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-08T17:18:42.321744+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:18:42.321744+0000","last_peered":"2024-05-08T17:18:42.321744+0000","last_clean":"2024-05-08T17:18:42.321744+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:18:42.321744+0000","last_undegraded":"2024-05-08T17:18:42.321744+0000","last_fullsized":"2024-05-08T17:18:42.321744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":56,"reported_epoch":39,"state":"active+clean","last_fresh":"2024-05-08T17:18:41.925436+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:18:41.925436+0000","last_peered":"2024-05-08T17:18:41.925436+0000","last_clean":"2024-05-08T17:18:41.925436+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:18:41.925436+0000","last_undegraded":"2024-05-08T17:18:41.925436+0000","last_fullsized":"2024-05-08T17:18:41.925436+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":51,"reported_epoch":39,"state":"active+undersized","last_fresh":"2024-05-08T17:18:42.321745+0000","last_change":"2024-05-08T17:17:50.340462+0000","last_active":"2024-05-08T17:18:42.321745+0000","last_peered":"2024-05-08T17:18:42.321745+0000","last_clean":"2024-05-08T17:17:48.722325+0000","last_became_active":"2024-05-08T17:17:50.340462+0000","last_became_peered":"2024-05-08T17:17:50.340462+0000","last_unstale":"2024-05-08T17:18:42.321745+0000","last_undegraded":"2024-05-08T17:18:42.321745+0000","last_fullsized":"2024-05-08T17:17:50.339899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:34.098879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":220,"reported_epoch":38,"state":"peering","last_fresh":"2024-05-08T17:18:41.254038+0000","last_change":"2024-05-08T17:18:41.925899+0000","last_active":"2024-05-08T17:18:41.925899+0000","last_peered":"2024-05-08T17:18:41.254038+0000","last_clean":"2024-05-08T17:18:41.254038+0000","last_became_active":"2024-05-08T17:18:35.799566+0000","last_became_peered":"2024-05-08T17:18:35.799566+0000","last_unstale":"2024-05-08T17:18:41.254038+0000","last_undegraded":"2024-05-08T17:18:41.254038+0000","last_fullsized":"2024-05-08T17:18:41.254038+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":36,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"no scrub is scheduled","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[0,6],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542175,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":141428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542173,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27352,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344488,"statfs":{"total":96636764160,"available":96608755712,"internally_reserved":0,"allocated":335872,"data_stored":141428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542173,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27360,"kb_used_data":340,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344480,"statfs":{"total":96636764160,"available":96608747520,"internally_reserved":0,"allocated":348160,"data_stored":145816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574876,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":670356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574877,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343968,"statfs":{"total":96636764160,"available":96608223232,"internally_reserved":0,"allocated":868352,"data_stored":670356,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":188416,"data_stored":67460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574877,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":328,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":335872,"data_stored":141428,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574876,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27340,"kb_used_data":320,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344500,"statfs":{"total":96636764160,"available":96608768000,"internally_reserved":0,"allocated":327680,"data_stored":136985,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:18:47.716 INFO:tasks.thrashosds.thrasher:Setting 2.7 to [2, 3] 2024-05-08T17:18:47.716 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.7', '2', '3'] 2024-05-08T17:18:47.717 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.7 2 3 2024-05-08T17:18:47.717 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:47.715+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:47.813 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:47.816+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:47.815 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:18:47.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:47.865 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:47.865 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:47.865 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:47.865 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:47.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:47.919 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:47.921+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:47.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2024-05-08T17:18:48.009 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:48.010 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:48.015 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:48.017+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:48.045 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:48.046 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:48.074 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.075 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.082 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:48.082 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:48.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:48.101 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:48.101 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:48.119 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:48.120 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:48.120 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:48.122+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:48.120 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:48.138 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:48.139 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:48.156 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:48.156 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:48.160 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:48.160 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:48.208 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.209 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.216 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:48.219+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:48.218 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:48.319 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:48.323+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:48.364 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.365 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.374 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:48.420 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:48.423+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:48.480 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.480 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:48.521 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:48.524+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:48.622 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:48.625+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:48.634 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.634 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.643 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:48.727 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:48.730+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:48.750 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.750 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.759 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:48.828 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:48.831+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:48.865 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:48.866 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:48.874 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:48.935 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:48.932+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:49.019 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:18:49.020 INFO:teuthology.orchestra.run.smithi154.stderr:set 2.7 pg_upmap mapping to [2,3] 2024-05-08T17:18:49.022 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2024-05-08T17:18:49.024 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.024 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:49.030 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:49.033+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:49.131 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:49.133+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:49.176 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.176 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.187 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:49.271 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:49.274+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:49.332 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:49.334+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:49.334 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.334 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.347 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:49.433 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:49.436+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:49.491 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.491 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.500 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:49.544 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:49.547+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:49.617 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.617 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.626 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:49.635 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:49.638+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:49.736 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:49.739+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:49.773 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.773 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.782 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:49.867 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:49.870+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:49.928 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:49.928 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:49.936 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:49.978 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:49.981+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:49.981 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2024-05-08T17:18:50.065 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:50.068+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:50.083 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:50.083 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:50.092 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:50.139 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:50.141+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:50.161 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:50.197 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:50.197 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:50.206 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:50.240 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:50.242+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:50.285 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:50.286 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:50.302 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:50.303 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:50.319 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:50.319 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:50.336 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:50.336 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:50.340 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:50.343+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:50.351 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:50.352 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:50.354 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:50.354 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:50.354 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:50.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:50.371 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:50.371 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:50.388 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:50.389 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:50.391 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:50.392 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:50.441 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:50.444+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:50.467 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:50.467 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:50.467 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:50.467 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:50.476 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:50.543 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:50.546+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:50.621 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:50.621 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:50.621 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:50.622 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:50.622 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:50.630 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:50.643 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:50.646+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:50.735 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.93501352800001, 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.736 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.737 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.738 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:50.739 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.740 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.741 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:50.742 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.90204993699999, 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.743 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.744 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:50.745 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:50.746 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.747 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.748 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.884285522, 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:50.749 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.750 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.751 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.752 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.753 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:50.754 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.88419249200001, 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.755 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:50.756 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:50.757 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:50.758 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.759 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.85609446699999, 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.760 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.761 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:50.762 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.763 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:50.764 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.765 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.766 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.85598439, 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.767 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:50.768 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:50.769 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:50.770 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:50.771 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.832123862, 2024-05-08T17:18:50.772 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.773 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.774 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.775 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:50.778 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:50.778 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.778 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.778 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.779 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:50.780 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.781 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.69076425399999, 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.782 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.783 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.784 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.785 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:50.786 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.69064705900001, 2024-05-08T17:18:50.787 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.788 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.789 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:50.790 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:50.791 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.792 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.69062210800001, 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.793 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.794 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.795 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:50.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:50.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:50.799 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.29275125000001, 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.802 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:50.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:50.806 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.29273567000001, 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:50.807 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:50.808 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:50.809 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.810 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.811 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.29271635000001, 2024-05-08T17:18:50.812 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:50.813 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:50.814 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.815 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:50.816 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:50.817 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:50.818 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.292639847, 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:50.819 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.820 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:50.821 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.822 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.823 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.29261825200001, 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:50.824 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.825 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:50.826 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:50.827 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:50.828 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.829 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.29259185800001, 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.830 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:50.831 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:50.832 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:50.833 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:50.834 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 136.292517632, 2024-05-08T17:18:50.835 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.836 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.837 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.838 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.839 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.840 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 135.806724135, 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:50.841 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:50.842 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:50.843 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:50.844 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.845 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.846 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 135.80671084799999, 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 135.80664178000001, 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:50.858 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:50.859 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:50.860 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:50.862+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:50.960 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:50.963+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:50.972 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:50.972 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:50.972 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:50.972 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:50.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2024-05-08T17:18:50.986 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:51.022 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:18:51.061 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:51.064+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:51.133 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.133 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:51.133 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:51.133 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:51.134 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.144 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:51.161 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:51.164+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.451987387, 2024-05-08T17:18:51.255 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.256 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:51.257 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.258 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:51.259 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.43001423499999, 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.260 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.261 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.262 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.263 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.42955787899999, 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.264 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:51.265 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:51.266 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:51.267 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.41051612300001, 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.268 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.269 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.270 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.271 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.272 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.40222706399999, 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.273 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.274 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.275 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.276 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.40173906000001, 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.277 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.278 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:51.279 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:51.280 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.281 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.40126462699999, 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.282 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:51.283 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.284 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:51.285 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.400871252, 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.286 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.287 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.288 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:51.289 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.39468481500001, 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:51.290 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.291 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.292 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:51.293 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.294 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.38988136399999, 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.295 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:51.296 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:51.297 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:51.298 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.38730590599999, 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.299 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.300 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.301 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:51.302 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.38452021000001, 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:51.303 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.304 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:51.305 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.306 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.307 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.37951758700001, 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.308 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.309 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:51.310 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.311 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 137.208341526, 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:51.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.313 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.314 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.315 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.79775755599999, 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.316 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:51.317 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:51.318 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:51.319 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.32496044499999, 2024-05-08T17:18:51.320 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.321 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.322 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.323 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:51.324 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.324374879, 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.325 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.326 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.327 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.32393547199999, 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:51.328 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.329 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.330 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:51.331 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.332 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.32328748699999, 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.333 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:51.334 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:51.335 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.336 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 136.31782576000001, 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:51.337 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:51.338 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.339 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.340 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.341 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.343 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:51.343 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:51.345+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:51.443 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:51.446+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:51.488 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.488 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:51.488 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:51.488 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:51.544 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:51.547+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:51.602 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.602 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:51.603 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:51.603 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:51.603 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.611 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:51.644 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:51.647+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:51.717 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.439101121999997, 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.718 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:18:51.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:18:51.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 1.377097228, 2024-05-08T17:18:51.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:18:51.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:18:51.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.726 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.727 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.727 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:51.727 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:51.727 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.728 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:51.745 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:51.749+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:51.834 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.834 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:51.834 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:51.834 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.843 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:51.886 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:51.889+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:51.947 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:51.950+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:51.949 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:51.949 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:51.949 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:51.949 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:51.949 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:51.957 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:51.975 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:18:51.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2024-05-08T17:18:52.048 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:52.051+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:52.083 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:52.083 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 63.804884827999999, 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:52.084 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:52.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:52.087 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:52.088 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:52.088 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:52.088 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:52.088 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:52.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:52.149 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:52.152+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:52.203 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:52.203 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:52.212 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:52.257 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:52.259+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:52.318 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:52.318 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:52.318 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:52.318 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:52.318 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:52.327 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:52.368 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:52.371+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:52.393 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:52.454 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:52.457+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:52.475 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:52.475 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:52.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:52.475 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:52.475 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:52.484 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:52.534 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:52.535 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:52.552 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:52.552 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:52.552 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:52.555+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:52.569 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:52.569 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:52.585 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:52.585 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:52.603 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:52.603 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:52.603 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:52.604 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:52.612 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:52.620 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:52.620 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:52.637 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:52.637 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:52.641 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:52.641 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:52.652 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:52.655+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:52.754 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:52.757+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:52.757 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:52.757 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:52.766 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:52.855 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:52.858+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:52.871 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:52.871 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:52.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:52.955 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:52.958+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:52.983 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2024-05-08T17:18:52.986 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:52.986 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:52.995 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:53.067 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:53.070+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:53.140 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.141 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.149 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:53.193 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:53.196+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:53.254 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.254 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.258 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:53.260+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:53.263 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:53.359 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:53.361+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:53.408 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.408 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.417 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:53.460 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:53.463+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:53.521 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.521 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.530 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:53.574 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:53.577+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:53.635 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.635 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.644 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:53.687 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:53.690+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:53.750 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.750 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.760 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:53.763 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:53.766+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:53.864 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:53.867+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:53.906 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:53.906 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:53.915 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:53.965 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:53.968+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:53.987 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2024-05-08T17:18:53.991 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:18:54.024 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:54.024 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:54.025 INFO:tasks.thrashosds.thrasher:inject_pause on osd.2 2024-05-08T17:18:54.025 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:18:54.025 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:18:54.025 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok config set filestore_inject_stall 3 2024-05-08T17:18:54.067 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.069 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.072 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:54.074+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:54.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:54.150 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:54.150 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T17:18:54.150 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:54.206 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:54.208+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:54.238 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.238 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.246 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:54.288 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:54.290+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:54.369 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:54.372+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:54.393 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.394 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.403 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:54.471 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:54.474+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:54.508 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.509 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.519 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:54.572 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:54.574+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:54.626 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.626 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.635 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:54.642 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:54.674 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:54.677+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:54.774 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:54.778+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:54.788 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:54.788 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:54.798 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:54.805 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:54.805 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:54.823 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:54.823 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:54.855 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:54.855 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:54.872 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:54.872 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:54.875 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:54.878+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:54.890 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:54.891 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:54.891 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:54.908 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:54.909 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 66.637679997999996, 2024-05-08T17:18:54.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:54.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:18:54.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:18:54.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:54.921 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:54.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:54.926 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:54.927 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:54.930 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:54.930 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:54.972 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:18:54.976 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:54.979+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:54.996 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2024-05-08T17:18:55.036 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:55.036 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:55.036 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:55.036 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:55.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:55.077 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:55.080+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:55.179 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:55.181+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:55.196 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:55.196 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:55.196 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:55.196 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:55.196 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:55.206 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:55.279 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:55.282+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:55.312 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:55.312 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:55.312 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:55.312 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:18:55.313 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:55.321 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:55.380 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:55.382+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:55.439 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:55.439 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:55.439 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:55.439 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:55.448 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:55.481 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:55.484+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:55.553 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:55.554 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:55.554 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:55.563 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:55.606 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:55.609+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:55.667 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:55.667 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:55.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:55.668 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:18:55.668 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:55.677 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:55.718 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:55.721+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:55.783 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:55.786+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:55.822 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:55.822 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:55.822 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:55.822 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:55.831 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:55.884 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:55.887+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:55.935 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:55.935 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:55.935 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:55.944 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:55.988 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:55.991+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:55.990 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.857915145, 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.050 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.051 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.052 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:18:56.053 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 21.854879626999999, 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:18:56.054 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:18:56.055 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:18:56.056 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.057 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.058 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:56.059 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:56.085 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:56.088+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:56.186 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:56.189+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:56.205 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:56.205 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:18:56.205 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:56.214 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:56.287 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:56.289+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:56.359 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:56.359 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:18:56.359 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:18:56.359 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:18:56.359 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:56.368 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:56.387 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:56.390+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:56.474 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:56.474 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:56.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:56.474 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.673727716, 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:56.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:56.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:56.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:56.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.640764125, 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.481 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:56.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:56.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.486 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.62299970999999, 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.62290668, 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.492 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:56.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:56.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:56.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.497 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.59480865500001, 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:56.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:56.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:56.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:56.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:56.502 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.59469857799999, 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.503 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:56.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:56.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:56.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:56.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.57083804999999, 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.508 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:56.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:56.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.429478442, 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.514 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:56.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.519 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.429361247, 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:56.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:56.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:56.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:56.524 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.429336296, 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.530 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.031465438, 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:56.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:56.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:56.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:56.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:56.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.031449858, 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:56.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.031430538, 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:56.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:56.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:56.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.03135403499999, 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:56.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:56.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:56.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:56.552 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.03133244, 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:56.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:56.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:56.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:56.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.031306046, 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.558 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:56.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:56.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 142.03123181999999, 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:56.563 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 141.54543832300001, 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 141.54542503600001, 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:56.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:56.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:56.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:56.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 141.545355968, 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:56.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:56.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:56.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:56.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:56.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:56.585 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:56.586 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:56.587 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:56.589+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:56.687 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:56.690+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:56.693 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:56.693 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:56.693 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:56.693 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:56.701 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:56.789 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:56.792+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:56.847 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:56.847 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:56.847 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:56.847 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:56.847 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:56.856 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:56.889 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:56.892+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:56.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:18:56.965 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:56.965 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:18:56.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:18:56.965 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:18:56.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.162165706, 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:56.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:18:56.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:18:56.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:56.970 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.14019255400001, 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:18:56.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:18:56.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:18:56.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.13973619800001, 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.978 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.120694442, 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.979 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:18:56.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:18:56.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.983 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.11240538300001, 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:18:56.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:18:56.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.111917379, 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.988 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.992 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.11144294600001, 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:56.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:18:56.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.998 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.111049571, 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:56.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:18:57.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.104863134, 2024-05-08T17:18:57.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:18:57.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.10005968300001, 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.008 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:18:57.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:18:57.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.09748422499999, 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.013 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:57.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.094698529, 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:57.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 143.089695906, 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:18:57.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:18:57.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:18:57.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.91851984499999, 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:18:57.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:18:57.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:57.030 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.50793587499999, 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:18:57.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.034 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.03513876400001, 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:18:57.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:18:57.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.034553198, 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.039 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:18:57.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:18:57.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:57.043 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.03411379100001, 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:18:57.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.047 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.03346580600001, 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:57.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:18:57.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:57.052 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 142.028004079, 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:57.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:18:57.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:18:57.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:57.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:57.057 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:57.057 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:18:57.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:18:57.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2024-05-08T17:18:57.060 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:57.060 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:57.061 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:57.060+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:57.075 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:57.075 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:57.094 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:57.095 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:57.112 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:57.113 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:57.131 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:57.131 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:57.131 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:57.149 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:57.150 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:57.159 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:57.162+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:57.167 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:57.168 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:57.172 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:57.173 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:57.186 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.187 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.197 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:18:57.260 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:57.263+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:57.345 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.345 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.354 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:18:57.399 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:57.402+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:57.460 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.461 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.461 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:57.465+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:57.469 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:18:57.563 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:57.566+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:57.615 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.615 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.625 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:18:57.663 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:57.667+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:57.765 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:57.768+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:57.771 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.771 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:18:57.865 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:57.869+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:57.925 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:57.926 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:57.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:18:57.976 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:57.979+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:57.977 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:18:58.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2024-05-08T17:18:58.068 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:58.071+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:58.086 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.086 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.096 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:18:58.168 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:58.172+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:58.244 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.244 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.254 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:18:58.269 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:58.272+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:58.362 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.362 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.370 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:58.374+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:58.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:18:58.471 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:58.475+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:58.488 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.488 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.498 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:18:58.572 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:58.575+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:58.603 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.603 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.612 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:18:58.674 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:58.677+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:58.757 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.757 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.766 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:18:58.810 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:18:58.813+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:18:58.871 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:58.871 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:58.876 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:58.879+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:58.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:18:58.977 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:58.980+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:59.001 INFO:tasks.mon_recovery:adding mon b back in 2024-05-08T17:18:59.001 INFO:tasks.ceph.mon.b:Restarting daemon 2024-05-08T17:18:59.001 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2024-05-08T17:18:59.003 INFO:tasks.ceph.mon.b:Started 2024-05-08T17:18:59.004 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:18:59.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:18:59.026 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:59.027 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:59.036 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:18:59.085 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:18:59.089+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:18:59.165 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:18:59.165 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:18:59.165 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap_items 2024-05-08T17:18:59.165 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T17:18:59.174 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:18:59.178 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:59.181+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:59.184 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:59.184 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:59.193 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:18:59.279 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:18:59.282+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:18:59.340 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:59.340 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:59.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:18:59.375 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:18:59.376 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:18:59.381 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:18:59.383+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:18:59.394 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:18:59.394 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:18:59.412 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:18:59.412 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:18:59.413 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":29,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:18:59.430 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:18:59.430 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:18:59.447 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:18:59.447 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:18:59.447 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:18:59.455 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:18:59.455 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:18:59.464 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:18:59.465 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:18:59.465 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:18:59.481 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:18:59.482 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:18:59.485 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:18:59.485 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 48, 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:16:22.335952+0000", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:18:57.980325+0000", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "last_up_change": "2024-05-08T17:17:49.336612+0000", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "last_in_change": "2024-05-08T17:18:41.921552+0000", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T17:18:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_num": 5801984, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_set": [ 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "noscrub", 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "pglog_hardlimit", 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "purged_snapdirs", 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_deletes", 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "sortbitwise" 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_version": 4, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_snapshot": "", 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_max": 2, 2024-05-08T17:18:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "max_osd": 8, 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "require_min_compat_client": "luminous", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "min_compat_client": "luminous", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "require_osd_release": "squid", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "allow_crimson": false, 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "pools": [ 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 1, 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": ".mgr", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:32.734319+0000", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:18:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 1, 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:18:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "16", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:18:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:18:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "options": { 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_max": 32, 2024-05-08T17:18:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_min": 1 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": {} 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 6.9000000953674316, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 6.9000000953674316, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:18:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 2, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": "rbd", 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:33.268021+0000", 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:18:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 8, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 8, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 8, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 8, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 8, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:18:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "17", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:18:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:18:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "options": {}, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd": {} 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 2.619999885559082, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 2.619999885559082, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.86000001430511475, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2.25, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2.25, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:18:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858", 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6808", 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6809", 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6811", 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6810", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6808/3590430155", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6809/3590430155", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6811/3590430155", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6810/3590430155", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "7e66ef3b-5581-4f8c-9807-e16ea4ec972e", 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 39, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6812", 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6813", 2024-05-08T17:18:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6815", 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6814", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6812/2114037860", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6813/2114037860", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6815/2114037860", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6814/2114037860", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "86378b92-f863-41ec-a3a5-f1dc78645250", 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 44, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6800", 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6801", 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6803", 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6802", 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6800/2790378686", 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6801/2790378686", 2024-05-08T17:18:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6803/2790378686", 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6802/2790378686", 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "2fa93019-16da-44c6-ac77-0ef554e24e10", 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6804", 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6805", 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6807", 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6806", 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6804/3602122863", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6805/3602122863", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6807/3602122863", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6806/3602122863", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "08750d7a-1ca6-4d68-a62e-893e8d979921", 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6812", 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6813", 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6815", 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6814", 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:18:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6812/1597208930", 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6813/1597208930", 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1597208930", 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6814/1597208930", 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:18:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "770b03a7-7f15-4427-81ce-e512b68bbb5c", 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 0, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 0, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 24, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6801", 2024-05-08T17:18:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6803", 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6807", 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6805", 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:18:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6801/3448847701", 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6803/3448847701", 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6807/3448847701", 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6805/3448847701", 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "exists" 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:18:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "ac82b5c6-7db8-4383-b0ac-0a203a4d84cd", 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6800", 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6802", 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6806", 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6804", 2024-05-08T17:18:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6800/3679914992", 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6802/3679914992", 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6806/3679914992", 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6804/3679914992", 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "813f2f3d-68eb-487b-89a1-8b9a5a103485", 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 42, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:18:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6808", 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6809", 2024-05-08T17:18:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6811", 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:18:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6810", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6808/2695818974", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6809/2695818974", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6811/2695818974", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6810/2695818974", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:18:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_xinfo": [ 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.817172+0000", 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.851798+0000", 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.765787+0000", 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.790359+0000", 2024-05-08T17:18:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.824178+0000", 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:18:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.696786+0000", 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 24 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.761200+0000", 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.764552+0000", 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap": [ 2024-05-08T17:18:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "pgid": "2.7", 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: 2, 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: 3 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_items": [], 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_primaries": [], 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_temp": [], 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_temp": [], 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklist": {}, 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "range_blocklist": {}, 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profiles": { 2024-05-08T17:18:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "default": { 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "crush-failure-domain": "osd", 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "k": "2", 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "m": "1", 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin": "jerasure", 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "technique": "reed_sol_van" 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps_queue": [], 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "new_removed_snaps": [], 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "new_purged_snaps": [], 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_node_flags": {}, 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "device_class_flags": {}, 2024-05-08T17:18:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": { 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_enabled": false, 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_bucket_count": 0, 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "degraded_stretch_mode": 0, 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "recovering_stretch_mode": 0, 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_bucket": 0 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:18:59.563 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:59.564 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:18:59.512+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:18:59.565 INFO:tasks.thrashosds.thrasher:j is {'epoch': 48, 'fsid': 'c69ba297-505d-4f4e-b390-bd22ef533cfc', 'created': '2024-05-08T17:16:22.335952+0000', 'modified': '2024-05-08T17:18:57.980325+0000', 'last_up_change': '2024-05-08T17:17:49.336612+0000', 'last_in_change': '2024-05-08T17:18:41.921552+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T17:16:32.734319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T17:16:33.268021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 3590430155}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6809', 'nonce': 3590430155}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6811', 'nonce': 3590430155}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 3590430155}]}, 'public_addr': '172.21.15.154:6808/3590430155', 'cluster_addr': '172.21.15.154:6809/3590430155', 'heartbeat_back_addr': '172.21.15.154:6811/3590430155', 'heartbeat_front_addr': '172.21.15.154:6810/3590430155', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7e66ef3b-5581-4f8c-9807-e16ea4ec972e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 2114037860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6813', 'nonce': 2114037860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6815', 'nonce': 2114037860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 2114037860}]}, 'public_addr': '172.21.15.154:6812/2114037860', 'cluster_addr': '172.21.15.154:6813/2114037860', 'heartbeat_back_addr': '172.21.15.154:6815/2114037860', 'heartbeat_front_addr': '172.21.15.154:6814/2114037860', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '86378b92-f863-41ec-a3a5-f1dc78645250', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 2790378686}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6801', 'nonce': 2790378686}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6803', 'nonce': 2790378686}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 2790378686}]}, 'public_addr': '172.21.15.154:6800/2790378686', 'cluster_addr': '172.21.15.154:6801/2790378686', 'heartbeat_back_addr': '172.21.15.154:6803/2790378686', 'heartbeat_front_addr': '172.21.15.154:6802/2790378686', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2fa93019-16da-44c6-ac77-0ef554e24e10', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 3602122863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6805', 'nonce': 3602122863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6807', 'nonce': 3602122863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 3602122863}]}, 'public_addr': '172.21.15.154:6804/3602122863', 'cluster_addr': '172.21.15.154:6805/3602122863', 'heartbeat_back_addr': '172.21.15.154:6807/3602122863', 'heartbeat_front_addr': '172.21.15.154:6806/3602122863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '08750d7a-1ca6-4d68-a62e-893e8d979921', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1597208930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6813', 'nonce': 1597208930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6815', 'nonce': 1597208930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1597208930}]}, 'public_addr': '172.21.15.183:6812/1597208930', 'cluster_addr': '172.21.15.183:6813/1597208930', 'heartbeat_back_addr': '172.21.15.183:6815/1597208930', 'heartbeat_front_addr': '172.21.15.183:6814/1597208930', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '770b03a7-7f15-4427-81ce-e512b68bbb5c', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6801', 'nonce': 3448847701}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6803', 'nonce': 3448847701}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6807', 'nonce': 3448847701}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6805', 'nonce': 3448847701}]}, 'public_addr': '172.21.15.183:6801/3448847701', 'cluster_addr': '172.21.15.183:6803/3448847701', 'heartbeat_back_addr': '172.21.15.183:6807/3448847701', 'heartbeat_front_addr': '172.21.15.183:6805/3448847701', 'state': ['exists']}, {'osd': 6, 'uuid': 'ac82b5c6-7db8-4383-b0ac-0a203a4d84cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 3679914992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 3679914992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 3679914992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 3679914992}]}, 'public_addr': '172.21.15.183:6800/3679914992', 'cluster_addr': '172.21.15.183:6802/3679914992', 'heartbeat_back_addr': '172.21.15.183:6806/3679914992', 'heartbeat_front_addr': '172.21.15.183:6804/3679914992', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '813f2f3d-68eb-487b-89a1-8b9a5a103485', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 2695818974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6809', 'nonce': 2695818974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6811', 'nonce': 2695818974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 2695818974}]}, 'public_addr': '172.21.15.183:6808/2695818974', 'cluster_addr': '172.21.15.183:6809/2695818974', 'heartbeat_back_addr': '172.21.15.183:6811/2695818974', 'heartbeat_front_addr': '172.21.15.183:6810/2695818974', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.817172+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.851798+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.765787+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.790359+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.824178+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.696786+0000', 'dead_epoch': 24}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.761200+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.764552+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.7', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T17:18:59.565 INFO:tasks.thrashosds.thrasher:No pg_upmap entries; doing nothing 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.77057826999999, 2024-05-08T17:18:59.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:18:59.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.73761467899999, 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:18:59.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:18:59.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.719850264, 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:18:59.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:18:59.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:18:59.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:18:59.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:18:59.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:59.589 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.71975723400001, 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:18:59.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:18:59.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:18:59.594 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.69165920899999, 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.595 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:59.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:18:59.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:18:59.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.69154913200001, 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.601 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:59.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:18:59.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:18:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.66768860400001, 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.607 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.52632899599999, 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:18:59.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.52621180099999, 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:18:59.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:18:59.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:18:59.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.52618684999999, 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:18:59.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:18:59.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:18:59.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12831599200001, 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12830041199999, 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:18:59.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:18:59.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.640 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12828109200001, 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.645 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12820458900001, 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.646 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:18:59.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:18:59.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:18:59.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12818299400001, 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.12815660000001, 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.657 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:18:59.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.661 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 145.128082374, 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.663 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:59.668 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 144.642288877, 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:59.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:18:59.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 144.64227559, 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:18:59.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:18:59.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:18:59.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 144.64220652200001, 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:18:59.679 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.684 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:18:59.685 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:18:59.685 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:18:59.686 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:18:59.687 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:59.689+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:59.787 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:18:59.790+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:18:59.792 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:59.792 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:59.793 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:18:59.793 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:59.803 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:18:59.888 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:18:59.892+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:18:59.910 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:18:59.910 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:18:59.911 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:18:59.911 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:18:59.911 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:18:59.919 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:18:59.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:19:00.003 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:00.007+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:00.070 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.070 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:00.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:00.070 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.266470414, 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:00.071 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:00.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:00.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:00.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:00.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:00.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:00.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.081 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.24449726200001, 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.082 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:00.087 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.088 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.089 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.24404090600001, 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.090 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:00.091 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.22499915, 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.094 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:00.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:00.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:00.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.21671009100001, 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:00.098 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:00.102 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.21622208700001, 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:00.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.21574765400001, 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:00.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:00.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:00.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.111 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.215354279, 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:00.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:00.117 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.209167842, 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:00.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:00.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:00.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.20436439100001, 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:00.122 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:00.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:00.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.128 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.20178893299999, 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:00.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.199003237, 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:00.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.194000614, 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.141 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:00.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:00.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.149 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 146.02282455299999, 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.150 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:00.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:00.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:00.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:00.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.159 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.61224058299999, 2024-05-08T17:19:00.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:00.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:00.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:00.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:00.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:00.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.13944347200001, 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:00.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:00.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.138857906, 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:00.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:00.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:00.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.13841849900001, 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:00.176 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.13777051400001, 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 145.132308787, 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:00.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:00.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:00.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:00.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.192 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.192 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.192 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.192 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.192 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.194 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:00.195 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:00.197+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:00.294 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:00.298+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:00.303 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.303 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:00.303 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:00.303 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.312 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:00.396 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:00.400+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:00.418 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.418 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:00.418 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:00.418 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:00.418 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.427 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:00.510 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:00.514+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:00.571 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 72.293091615999998, 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:00.572 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:00.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.576 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 10.231087722, 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:00.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:00.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.580 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.581 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.581 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.582 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:00.623 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:00.626+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:00.699 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:00.702+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:00.728 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.728 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:00.728 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:00.728 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.737 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:00.800 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:00.803+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:00.843 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.843 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:00.843 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:00.843 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:00.844 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.852 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:00.901 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:00.904+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 72.679286253000001, 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:00.958 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:00.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:00.962 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:00.966 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:00.991 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:19:01.006 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:01.009+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:01.097 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:01.097 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:01.097 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:01.102 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:01.106+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:01.106 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:01.204 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:01.207+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:01.251 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:01.252 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:01.252 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:01.252 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:01.252 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:01.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:01.344 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:01.348+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:01.406 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:01.409+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:01.407 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:01.407 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:01.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:01.407 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:01.407 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:01.416 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:01.486 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:01.506 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:01.509+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:01.533 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:01.534 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:01.534 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:01.534 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:01.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:01.607 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:01.610+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:01.607 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:01.608 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:01.625 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:01.625 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:01.641 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:01.642 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:01.658 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:01.658 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:01.675 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:01.676 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:01.676 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:01.690 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:01.690 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:01.692 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:01.693 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:01.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:01.708 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:01.711+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:01.710 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:01.710 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:01.713 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:01.714 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:01.803 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:01.804 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:01.809 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:01.812+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:01.812 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:01.910 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:01.913+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:01.917 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:01.917 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:01.925 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:02.011 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:02.014+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:02.030 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.031 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.039 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:02.112 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:02.115+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:02.145 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.145 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.154 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:02.213 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:02.217+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:02.258 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.259 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.267 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:02.314 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:02.317+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:02.372 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.373 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.382 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:02.426 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:02.429+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:02.429 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:02.487 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.488 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.496 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:02.537 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:02.540+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:02.608 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.608 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:02.618 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:02.621+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:02.719 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:02.722+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:02.766 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.766 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.775 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:02.803 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":32,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:02.819 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:02.823+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:02.883 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:02.883 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:02.892 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:02.935 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:02.939+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:03.000 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.000 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.006 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:19:03.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:03.022 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:03.024+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:03.123 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:03.126+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:03.159 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.159 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:03.224 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:03.227+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:03.274 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.274 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.283 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:03.325 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:03.328+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:03.389 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.389 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.397 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:03.426 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:03.430+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:03.502 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.502 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:03.554 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:03.558+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:03.618 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.618 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.627 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:03.669 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:03.672+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:03.715 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:03.730 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:03.733+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:03.776 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:03.776 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:03.786 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:03.830 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:03.834+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:03.840 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:03.840 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:03.858 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:03.859 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:03.875 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:03.875 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:03.893 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:03.894 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:03.894 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:03.894 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:03.894 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:03.894 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:03.895 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:03.904 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:03.910 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:03.911 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:03.911 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:03.929 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:03.929 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:03.931 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:03.935+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:03.947 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:03.947 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:03.950 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:03.951 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:04.005 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:19:04.020 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:04.020 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:04.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:04.020 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:04.020 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:04.029 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:04.071 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:04.074+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:04.133 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:04.137+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:04.145 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.146 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:04.146 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:04.146 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.155 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:04.235 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:04.237+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:04.260 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:04.260 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:04.260 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.268 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:04.335 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:04.338+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:04.373 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.373 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:04.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:04.373 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:04.373 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.382 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:04.466 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:04.469+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:04.527 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:04.528 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:04.528 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.536 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:04.537 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:04.539+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:04.566 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:04.566 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:04.637 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:04.641+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:04.681 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.681 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:04.682 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:04.682 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.690 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:04.738 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:04.741+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.604109000000001, 2024-05-08T17:19:04.796 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.797 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:04.798 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:04.799 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 30.601073482, 2024-05-08T17:19:04.800 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.801 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:04.802 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.803 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:04.804 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:04.805 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.806 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:04.848 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:04.851+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:04.940 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:04.943+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:04.953 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:04.953 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:04.953 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:04.953 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:04.962 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:05.041 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:05.045+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:05.107 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:05.107 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:05.107 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:05.118 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:05.142 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:05.146+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:05.224 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:05.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.423859349, 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:05.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.230 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.390895758, 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.231 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.232 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:05.233 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:05.234 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:05.235 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.236 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.237 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.37313134300001, 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.238 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:05.239 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.240 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.241 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.242 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.243 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.373038313, 2024-05-08T17:19:05.244 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.245 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.246 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.247 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.248 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.249 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.344940288, 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.250 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.251 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.252 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.253 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.254 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.34483021099999, 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:05.255 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.256 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.257 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.258 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:05.259 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.260 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.32096968299999, 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.261 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.262 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:05.263 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.264 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.265 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.179610075, 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:05.266 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.267 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.268 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.269 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.270 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:05.271 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.17949288, 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.272 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:05.273 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:05.274 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:05.275 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.276 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 151.179467929, 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.277 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:05.278 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:05.279 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.280 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.281 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.282 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.78159707099999, 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.283 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:05.284 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.285 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.286 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.287 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.781581491, 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.288 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:05.289 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.290 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.291 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.292 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.78156217099999, 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.293 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.294 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.295 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:05.296 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.297 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.298 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.78148566799999, 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:05.299 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:05.300 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:05.301 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:05.302 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:05.303 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.781464073, 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.304 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.305 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.306 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.307 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.308 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.78143767899999, 2024-05-08T17:19:05.309 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.310 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.311 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.312 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.313 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.314 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.78136345300001, 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.315 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.316 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.317 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.318 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.319 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.29556995600001, 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.320 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:05.321 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:05.322 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:05.323 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:05.324 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.325 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.29555666900001, 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.326 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.327 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.328 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:05.329 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.330 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 150.29548760099999, 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:05.331 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:05.332 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:05.333 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:05.334 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:05.335 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:05.336 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:05.337 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:05.341+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:05.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:05.439 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:05.442+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:05.455 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:05.455 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:05.455 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:05.455 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:05.464 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:05.540 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:05.543+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:05.571 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:05.571 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:05.571 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:05.571 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:05.572 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:05.580 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:05.641 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:05.644+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.88496905700001, 2024-05-08T17:19:05.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:05.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:05.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.692 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.86299590499999, 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.86253954899999, 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.697 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:05.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:05.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:05.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.84349779300001, 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:05.701 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:05.705 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.83520873399999, 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.83472072999999, 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.83424629699999, 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.714 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:05.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.83385292200001, 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.82766648500001, 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:05.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:05.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:05.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:05.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.82286303399999, 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.820287576, 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.81750188000001, 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:05.737 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.81249925700001, 2024-05-08T17:19:05.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:05.745 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.641323196, 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:05.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.749 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 151.230739226, 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:05.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:05.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:05.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:05.753 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 150.75794211499999, 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:05.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.758 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 150.75735654900001, 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.762 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 150.75691714199999, 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.764 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.765 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.766 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 150.75626915699999, 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:05.767 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:05.768 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:05.769 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.770 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 150.75080743000001, 2024-05-08T17:19:05.771 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:05.772 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:05.773 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:05.774 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:05.775 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:05.777 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:05.777 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:05.780+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:05.817 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:05.877 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:05.881+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:05.927 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:05.927 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:05.927 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:05.927 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:05.936 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:05.952 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:05.978 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:05.981+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:06.020 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:19:06.081 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:06.082+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:06.092 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:06.092 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:06.092 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:06.092 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:06.092 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:06.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:06.113 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:06.113 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:06.133 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:06.133 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:06.152 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:06.152 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:06.169 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:06.170 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:06.180 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:06.183+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:06.188 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:06.188 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:06.188 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:06.194 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":36,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:06.206 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:06.207 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:06.224 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.225 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.225 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:06.225 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:06.228 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:06.228 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:06.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:06.281 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:06.285+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:06.380 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.381 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.382 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:06.385+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:06.389 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:06.483 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:06.487+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.495 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:06.585 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:06.588+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:06.609 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.609 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.618 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:06.685 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:06.689+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:06.723 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.724 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.733 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:06.786 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:06.790+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:06.838 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.838 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:06.847 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:06.887 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:06.891+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:06.988 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:06.992+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:06.992 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:06.992 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.001 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:07.022 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:19:07.089 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:07.092+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:07.118 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.118 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.127 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:07.190 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:07.193+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:07.272 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.272 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:07.324 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:07.327+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:07.385 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.385 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.393 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:07.396+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:07.394 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:07.494 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:07.497+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:07.539 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.539 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.548 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:07.595 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:07.598+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:07.653 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.653 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.662 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:07.696 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:07.700+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:07.797 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:07.799+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:07.808 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.808 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.816 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:07.900 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:07.903+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:07.960 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:07.961 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:07.969 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:08.010 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:08.013+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:08.100 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:08.103+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:08.114 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:08.114 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:08.123 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:08.207 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:08.210+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:08.230 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:08.267 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:08.267 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:08.276 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:08.317 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:08.320+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:08.367 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:08.367 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:08.385 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:08.385 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:08.390 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:08.391 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:08.393 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:08.393 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:08.400 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:08.403 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:08.403 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:08.403 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:08.406+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:08.420 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:08.420 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:08.438 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:08.439 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:08.439 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:08.456 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:08.457 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:08.473 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:08.474 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:08.477 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:08.478 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:08.503 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:08.507+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:08.509 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:08.509 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:08.509 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:08.518 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:08.604 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:08.608+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:08.666 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.863007135, 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:08.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:08.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:08.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.84103398299999, 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:08.671 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.676 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.84057762699999, 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:08.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:08.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:08.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.82153587100001, 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:08.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:08.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:08.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.81324681199999, 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:08.686 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:08.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:08.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.81275880800001, 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.691 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:08.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:08.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:08.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.81228437499999, 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:08.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.811891, 2024-05-08T17:19:08.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:08.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:08.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:08.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:08.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.706 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.80570456300001, 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:08.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:08.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:08.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.710 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.80090111199999, 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:08.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:08.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.714 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:08.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.798325654, 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.717 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:08.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:08.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:08.721 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.79553995800001, 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:08.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.79053733500001, 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.727 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.619361274, 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.730 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:08.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:08.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 154.20877730399999, 2024-05-08T17:19:08.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:08.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.738 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 153.73598019299999, 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:08.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:08.743 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 153.73539462700001, 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:08.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:08.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 153.73495521999999, 2024-05-08T17:19:08.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:08.750 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:08.752 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 153.73430723499999, 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:08.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:08.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 153.72884550800001, 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:08.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.760 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.761 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.761 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:08.761 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:08.761 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:08.762 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:08.763 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:08.765+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:08.863 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:08.867+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:08.909 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:08.909 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:08.909 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:08.909 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:08.918 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:08.964 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:08.967+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:09.025 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:09.025 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:09.025 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.034 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:09.035 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:19:09.065 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:09.068+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:09.145 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 80.866842027000004, 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 18.804838133000001, 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:09.156 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.158 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:09.201 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:09.205+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:09.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:09.266 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:09.269+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:09.266 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.267 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:09.267 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:09.267 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.276 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:09.368 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:09.371+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:09.423 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.423 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:09.424 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:09.424 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:09.424 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.433 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:09.476 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:09.480+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 81.261247349000001, 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:09.544 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.549 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:09.567 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:09.567 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:09.569 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:09.573+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:09.575 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":38,"quorum":[0,2],"quorum_names":["a","c"],"quorum_leader_name":"a","quorum_age":39,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:09.670 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:09.673+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:09.681 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.681 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:09.681 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:09.681 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:09.774 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:09.778+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:09.837 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.837 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:09.837 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:09.837 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:09.837 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:09.846 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:09.871 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:09.875+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:09.973 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:09.976+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:09.992 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:09.992 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:09.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:09.993 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:09.993 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:10.002 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:10.022 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:19:10.074 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:10.077+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:10.118 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:10.119 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:10.119 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:10.119 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:10.128 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:10.212 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:10.215+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:10.273 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:10.274 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:10.274 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:10.274 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:10.274 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:10.276 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:10.280+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:10.282 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:10.377 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:10.381+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:10.427 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:10.428 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:10.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:10.428 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:10.428 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:10.437 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:10.478 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:10.481+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:10.479 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:10.543 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:10.543 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:10.543 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:10.543 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:10.551 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:10.578 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:10.582+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:10.605 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:10.606 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:10.622 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:10.623 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:10.639 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:10.639 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:10.656 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:10.665 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:10.672 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:10.672 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:10.672 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:10.679 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:10.683+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:10.689 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:10.689 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:10.706 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:10.706 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:10.709 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:10.710 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:10.770 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:10.771 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:10.779 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:10.780 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:10.784+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:10.882 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:10.885+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:10.884 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:10.884 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:10.893 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:10.982 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:10.985+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:11.039 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.039 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.048 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:11.089 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:11.092+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:11.184 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:11.188+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:11.194 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.194 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.203 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:11.284 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:11.288+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:11.313 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.313 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.322 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:11.406 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:11.410+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:11.468 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.468 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.477 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:11.518 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:11.521+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:11.587 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:11.591+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:11.622 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.623 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:11.688 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:11.692+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:11.777 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.778 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.787 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:11.789 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:11.793+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:11.890 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:11.893+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:11.892 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:11.892 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:11.901 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:11.990 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:11.994+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:12.038 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:19:12.050 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.050 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.060 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:12.101 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:12.104+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:12.193 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:12.196+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:12.212 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.212 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.221 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:12.293 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:12.297+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:12.328 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.328 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.337 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:12.394 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:12.398+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:12.483 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.483 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.492 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:12.495 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:12.499+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:12.590 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:12.596 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:12.600+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:12.598 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.598 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.608 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:12.696 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:12.700+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:12.711 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:12.761 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.761 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.770 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:12.815 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:12.819+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:12.857 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:12.858 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:12.876 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:12.876 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:12.881 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:12.881 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:12.890 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:12.894 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:12.895 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:12.911 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:12.911 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:12.929 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:12.929 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:12.929 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:12.934 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:12.938+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:12.947 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:12.947 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:12.981 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:12.994 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:19:12.994 INFO:tasks.mon_recovery:removing mon c 2024-05-08T17:19:12.994 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T17:19:12.995 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:19:13.000 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:13.004+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:13.008 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.008 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.008 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:13.008 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.017 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:13.050 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T17:19:13.050 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 2 2024-05-08T17:19:13.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:13.101 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:13.105+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:13.165 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.165 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.165 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:13.165 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:13.165 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.166 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:13.166 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:13.170 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:13.171 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:13.174 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:13.201 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:13.205+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:13.303 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:13.306+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:13.319 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:13.320 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:13.320 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.328 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:13.404 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:13.407+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:13.433 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.433 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.433 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:13.433 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.442 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:13.444 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":2,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:13.505 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:13.509+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:13.546 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.547 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.547 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:13.547 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:13.547 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.555 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:13.606 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:13.609+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.469351134, 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:13.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 39.466315616000003, 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:13.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:13.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:13.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:13.668 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:13.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:13.669 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:13.669 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:13.669 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.670 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:13.714 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:13.718+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:13.775 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.775 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.775 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:13.775 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.783 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:13.808 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:13.811+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:13.910 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:13.913+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:13.928 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:13.928 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:13.928 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:13.928 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:13.928 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:13.937 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:14.010 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:14.014+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:14.085 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:14.085 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.28479228, 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.086 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.087 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.088 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.089 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.090 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.092 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.25182868900001, 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:14.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:14.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.095 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.23406427399999, 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.098 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.099 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.100 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.101 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.102 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:14.103 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.233971244, 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.104 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:14.105 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.106 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:14.107 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.108 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.20587321900001, 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:14.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:14.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.205763142, 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:14.115 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.117 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:14.118 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.119 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:14.121 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.18190261399999, 2024-05-08T17:19:14.122 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.123 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.124 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:14.125 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.04054300600001, 2024-05-08T17:19:14.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:14.130 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:14.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.04042581100001, 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.134 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:14.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:14.136 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.137 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.138 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:14.139 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 160.04040086000001, 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.140 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.141 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:14.142 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:14.143 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:14.144 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.145 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.642530002, 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.146 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.147 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.148 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.149 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.150 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:14.151 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.642514422, 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.152 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:14.153 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.154 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.155 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:14.156 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.642495102, 2024-05-08T17:19:14.157 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.158 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:14.159 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:14.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:14.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:14.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.642418599, 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.163 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:14.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:14.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:14.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:14.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.168 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.642397004, 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.171 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:14.172 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:14.173 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.64237061, 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.174 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.175 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.176 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.177 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.178 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.179 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.64229638399999, 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.180 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.181 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:14.182 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:14.183 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:14.184 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.185 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.15650288699999, 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.186 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.187 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.188 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:14.189 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.190 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.15648959999999, 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.191 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.192 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.193 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:14.194 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.195 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.196 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 159.156420532, 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:14.197 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:14.198 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.199 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.200 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:14.201 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.202 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:14.203 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:14.203 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:14.204 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:14.207+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:14.204 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:14.305 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:14.308+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:14.315 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:14.315 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:14.315 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.324 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:14.405 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:14.408+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:14.470 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.470 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:14.471 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:14.471 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:14.471 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.479 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:14.524 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:14.528+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:14.568 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:14.568 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:14.568 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T17:19:14.569 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.78472429999999, 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.588 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:14.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.76275114800001, 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:14.592 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.76229479200001, 2024-05-08T17:19:14.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.743253036, 2024-05-08T17:19:14.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:14.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:14.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.604 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.73496397700001, 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.734475973, 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:14.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:14.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:14.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.73400154000001, 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:14.614 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:14.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:14.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.618 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.73360816499999, 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:14.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:14.622 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.727421728, 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:14.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.72261827700001, 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.628 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:14.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:14.632 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.72004281900001, 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.717257123, 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.637 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:14.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:14.641 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.7122545, 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.645 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.54107843899999, 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 160.13049446900001, 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:14.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:14.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:14.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 159.65769735800001, 2024-05-08T17:19:14.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:14.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 159.65711179199999, 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:14.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 159.65667238500001, 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.663 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 159.65602440000001, 2024-05-08T17:19:14.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:14.671 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 159.650562673, 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:14.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:14.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:14.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.676 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.678 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:14.678 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:14.681+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:14.778 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:14.781+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:14.826 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:14.826 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:14.826 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.835 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:14.864 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:19:14.864 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 52, 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:16:22.335952+0000", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:19:10.025514+0000", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "last_up_change": "2024-05-08T17:17:49.336612+0000", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "last_in_change": "2024-05-08T17:18:41.921552+0000", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": "noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_num": 5801984, 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_set": [ 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "noscrub", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "pglog_hardlimit", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "purged_snapdirs", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_deletes", 2024-05-08T17:19:14.865 INFO:teuthology.orchestra.run.smithi154.stdout: "sortbitwise" 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_version": 4, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_snapshot": "", 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_max": 2, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "max_osd": 8, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "require_min_compat_client": "luminous", 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "min_compat_client": "luminous", 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "require_osd_release": "squid", 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "allow_crimson": false, 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: "pools": [ 2024-05-08T17:19:14.866 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 1, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": ".mgr", 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:32.734319+0000", 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:19:14.867 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 1, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 1, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 1, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 1, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 1, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "16", 2024-05-08T17:19:14.868 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:19:14.869 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:19:14.870 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "options": { 2024-05-08T17:19:14.871 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_max": 32, 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_min": 1 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": {} 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 6.9000000953674316, 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 6.9000000953674316, 2024-05-08T17:19:14.872 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 2, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": "rbd", 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:33.268021+0000", 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:19:14.873 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:19:14.874 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 8, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 8, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 8, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 8, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 8, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "17", 2024-05-08T17:19:14.875 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:19:14.876 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:19:14.877 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "options": {}, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd": {} 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:19:14.878 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 2.619999885559082, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 2.619999885559082, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.86000001430511475, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2.25, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2.25, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:19:14.879 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858", 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.880 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6808", 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6809", 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.881 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6811", 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.882 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6810", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6808/3590430155", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6809/3590430155", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6811/3590430155", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6810/3590430155", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "7e66ef3b-5581-4f8c-9807-e16ea4ec972e", 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 39, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.884 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6812", 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6813", 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:14.885 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6815", 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.886 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6814", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6812/2114037860", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6813/2114037860", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6815/2114037860", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6814/2114037860", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "86378b92-f863-41ec-a3a5-f1dc78645250", 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 44, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.888 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6800", 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6801", 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.889 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6803", 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.890 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6802", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6800/2790378686", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6801/2790378686", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6803/2790378686", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6802/2790378686", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.891 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "2fa93019-16da-44c6-ac77-0ef554e24e10", 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.892 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6804", 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6805", 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.893 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6807", 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.894 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6806", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6804/3602122863", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6805/3602122863", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6807/3602122863", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6806/3602122863", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "08750d7a-1ca6-4d68-a62e-893e8d979921", 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.896 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6812", 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6813", 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.897 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6815", 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.898 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6814", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6812/1597208930", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6813/1597208930", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1597208930", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6814/1597208930", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.899 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "770b03a7-7f15-4427-81ce-e512b68bbb5c", 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 0, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 0, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 24, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6801", 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6803", 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6807", 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.902 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6805", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6801/3448847701", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6803/3448847701", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6807/3448847701", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6805/3448847701", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "exists" 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "ac82b5c6-7db8-4383-b0ac-0a203a4d84cd", 2024-05-08T17:19:14.903 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6800", 2024-05-08T17:19:14.904 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6802", 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6806", 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6804", 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.906 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6800/3679914992", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6802/3679914992", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6806/3679914992", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6804/3679914992", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "813f2f3d-68eb-487b-89a1-8b9a5a103485", 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:14.907 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 42, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6808", 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:14.908 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6809", 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6811", 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6810", 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6808/2695818974", 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6809/2695818974", 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6811/2695818974", 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6810/2695818974", 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_xinfo": [ 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:19:14.911 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.817172+0000", 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.912 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.851798+0000", 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.765787+0000", 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:19:14.913 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.790359+0000", 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.914 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.824178+0000", 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.696786+0000", 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 24 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:19:14.915 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.761200+0000", 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.764552+0000", 2024-05-08T17:19:14.916 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap": [ 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "pgid": "2.7", 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: 2, 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: 3 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_items": [], 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_primaries": [], 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_temp": [], 2024-05-08T17:19:14.917 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_temp": [], 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklist": {}, 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "range_blocklist": {}, 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profiles": { 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "default": { 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "crush-failure-domain": "osd", 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "k": "2", 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "m": "1", 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin": "jerasure", 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "technique": "reed_sol_van" 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps_queue": [], 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "new_removed_snaps": [], 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "new_purged_snaps": [], 2024-05-08T17:19:14.918 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_node_flags": {}, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "device_class_flags": {}, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": { 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_enabled": false, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_bucket_count": 0, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "degraded_stretch_mode": 0, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "recovering_stretch_mode": 0, 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_bucket": 0 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:14.919 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.921 INFO:tasks.thrashosds.thrasher:j is {'epoch': 52, 'fsid': 'c69ba297-505d-4f4e-b390-bd22ef533cfc', 'created': '2024-05-08T17:16:22.335952+0000', 'modified': '2024-05-08T17:19:10.025514+0000', 'last_up_change': '2024-05-08T17:17:49.336612+0000', 'last_in_change': '2024-05-08T17:18:41.921552+0000', 'flags': 'noscrub,sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5801984, 'flags_set': ['noscrub', 'pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T17:16:32.734319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T17:16:33.268021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.619999885559082, 'score_stable': 2.619999885559082, 'optimal_score': 0.8600000143051147, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 3590430155}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6809', 'nonce': 3590430155}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6811', 'nonce': 3590430155}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 3590430155}]}, 'public_addr': '172.21.15.154:6808/3590430155', 'cluster_addr': '172.21.15.154:6809/3590430155', 'heartbeat_back_addr': '172.21.15.154:6811/3590430155', 'heartbeat_front_addr': '172.21.15.154:6810/3590430155', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7e66ef3b-5581-4f8c-9807-e16ea4ec972e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 2114037860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6813', 'nonce': 2114037860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6815', 'nonce': 2114037860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 2114037860}]}, 'public_addr': '172.21.15.154:6812/2114037860', 'cluster_addr': '172.21.15.154:6813/2114037860', 'heartbeat_back_addr': '172.21.15.154:6815/2114037860', 'heartbeat_front_addr': '172.21.15.154:6814/2114037860', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '86378b92-f863-41ec-a3a5-f1dc78645250', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 2790378686}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6801', 'nonce': 2790378686}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6803', 'nonce': 2790378686}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 2790378686}]}, 'public_addr': '172.21.15.154:6800/2790378686', 'cluster_addr': '172.21.15.154:6801/2790378686', 'heartbeat_back_addr': '172.21.15.154:6803/2790378686', 'heartbeat_front_addr': '172.21.15.154:6802/2790378686', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2fa93019-16da-44c6-ac77-0ef554e24e10', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 3602122863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6805', 'nonce': 3602122863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6807', 'nonce': 3602122863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 3602122863}]}, 'public_addr': '172.21.15.154:6804/3602122863', 'cluster_addr': '172.21.15.154:6805/3602122863', 'heartbeat_back_addr': '172.21.15.154:6807/3602122863', 'heartbeat_front_addr': '172.21.15.154:6806/3602122863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '08750d7a-1ca6-4d68-a62e-893e8d979921', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1597208930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6813', 'nonce': 1597208930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6815', 'nonce': 1597208930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1597208930}]}, 'public_addr': '172.21.15.183:6812/1597208930', 'cluster_addr': '172.21.15.183:6813/1597208930', 'heartbeat_back_addr': '172.21.15.183:6815/1597208930', 'heartbeat_front_addr': '172.21.15.183:6814/1597208930', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '770b03a7-7f15-4427-81ce-e512b68bbb5c', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6801', 'nonce': 3448847701}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6803', 'nonce': 3448847701}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6807', 'nonce': 3448847701}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6805', 'nonce': 3448847701}]}, 'public_addr': '172.21.15.183:6801/3448847701', 'cluster_addr': '172.21.15.183:6803/3448847701', 'heartbeat_back_addr': '172.21.15.183:6807/3448847701', 'heartbeat_front_addr': '172.21.15.183:6805/3448847701', 'state': ['exists']}, {'osd': 6, 'uuid': 'ac82b5c6-7db8-4383-b0ac-0a203a4d84cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 3679914992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 3679914992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 3679914992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 3679914992}]}, 'public_addr': '172.21.15.183:6800/3679914992', 'cluster_addr': '172.21.15.183:6802/3679914992', 'heartbeat_back_addr': '172.21.15.183:6806/3679914992', 'heartbeat_front_addr': '172.21.15.183:6804/3679914992', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '813f2f3d-68eb-487b-89a1-8b9a5a103485', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 2695818974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6809', 'nonce': 2695818974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6811', 'nonce': 2695818974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 2695818974}]}, 'public_addr': '172.21.15.183:6808/2695818974', 'cluster_addr': '172.21.15.183:6809/2695818974', 'heartbeat_back_addr': '172.21.15.183:6811/2695818974', 'heartbeat_front_addr': '172.21.15.183:6810/2695818974', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.817172+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.851798+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.765787+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.790359+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.824178+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.696786+0000', 'dead_epoch': 24}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.761200+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.764552+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.7', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T17:19:14.921 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:19:14.922 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:14.924+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:14.943 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:14.943 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:14.943 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:14.943 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:14.943 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:14.953 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:15.037 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:15.041+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 86.839258740000005, 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:15.118 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:15.123 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 24.777254846000002, 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:15.124 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:15.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:15.127 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:15.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:15.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:15.128 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:15.128 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:15.129 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:15.129 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:15.132+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:15.171 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:15.229 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:15.232+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:15.257 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:19:15.257 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:19:15.271 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-05-08T17:19:14.731147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":220716,"kb_used_data":4556,"kb_used_omap":12,"kb_used_meta":216051,"kb_avail":754754004,"statfs":{"total":773094113280,"available":772868100096,"internally_reserved":0,"allocated":4665344,"data_stored":2511652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":12751,"internal_metadata":221236785},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001387"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":24,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.890951+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:19:10.890951+0000","last_peered":"2024-05-08T17:19:10.890951+0000","last_clean":"2024-05-08T17:19:10.890951+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:19:10.890951+0000","last_undegraded":"2024-05-08T17:19:10.890951+0000","last_fullsized":"2024-05-08T17:19:10.890951+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":82,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.531441+0000","last_change":"2024-05-08T17:16:34.861311+0000","last_active":"2024-05-08T17:19:10.531441+0000","last_peered":"2024-05-08T17:19:10.531441+0000","last_clean":"2024-05-08T17:19:10.531441+0000","last_became_active":"2024-05-08T17:16:34.861086+0000","last_became_peered":"2024-05-08T17:16:34.861086+0000","last_unstale":"2024-05-08T17:19:10.531441+0000","last_undegraded":"2024-05-08T17:19:10.531441+0000","last_fullsized":"2024-05-08T17:19:10.531441+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:38:45.802839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":92,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.890350+0000","last_change":"2024-05-08T17:18:42.931415+0000","last_active":"2024-05-08T17:19:10.890350+0000","last_peered":"2024-05-08T17:19:10.890350+0000","last_clean":"2024-05-08T17:19:10.890350+0000","last_became_active":"2024-05-08T17:18:42.931150+0000","last_became_peered":"2024-05-08T17:18:42.931150+0000","last_unstale":"2024-05-08T17:19:10.890350+0000","last_undegraded":"2024-05-08T17:19:10.890350+0000","last_fullsized":"2024-05-08T17:19:10.890350+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T19:03:16.446490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":92,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.531430+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:19:10.531430+0000","last_peered":"2024-05-08T17:19:10.531430+0000","last_clean":"2024-05-08T17:19:10.531430+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:19:10.531430+0000","last_undegraded":"2024-05-08T17:19:10.531430+0000","last_fullsized":"2024-05-08T17:19:10.531430+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":78,"reported_epoch":52,"state":"active+undersized","last_fresh":"2024-05-08T17:19:10.531424+0000","last_change":"2024-05-08T17:17:50.342570+0000","last_active":"2024-05-08T17:19:10.531424+0000","last_peered":"2024-05-08T17:19:10.531424+0000","last_clean":"2024-05-08T17:17:48.339260+0000","last_became_active":"2024-05-08T17:17:50.342570+0000","last_became_peered":"2024-05-08T17:17:50.342570+0000","last_unstale":"2024-05-08T17:19:10.531424+0000","last_undegraded":"2024-05-08T17:19:10.531424+0000","last_fullsized":"2024-05-08T17:17:50.341965+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T02:07:06.433312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1],"acting":[1],"avail_no_missing":["1"],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":83,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.890945+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:19:10.890945+0000","last_peered":"2024-05-08T17:19:10.890945+0000","last_clean":"2024-05-08T17:19:10.890945+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:19:10.890945+0000","last_undegraded":"2024-05-08T17:19:10.890945+0000","last_fullsized":"2024-05-08T17:19:10.890945+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":82,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.890350+0000","last_change":"2024-05-08T17:16:34.861764+0000","last_active":"2024-05-08T17:19:10.890350+0000","last_peered":"2024-05-08T17:19:10.890350+0000","last_clean":"2024-05-08T17:19:10.890350+0000","last_became_active":"2024-05-08T17:16:34.861465+0000","last_became_peered":"2024-05-08T17:16:34.861465+0000","last_unstale":"2024-05-08T17:19:10.890350+0000","last_undegraded":"2024-05-08T17:19:10.890350+0000","last_fullsized":"2024-05-08T17:19:10.890350+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:45:53.112975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":78,"reported_epoch":52,"state":"active+undersized","last_fresh":"2024-05-08T17:19:10.890960+0000","last_change":"2024-05-08T17:17:50.340462+0000","last_active":"2024-05-08T17:19:10.890960+0000","last_peered":"2024-05-08T17:19:10.890960+0000","last_clean":"2024-05-08T17:17:48.722325+0000","last_became_active":"2024-05-08T17:17:50.340462+0000","last_became_peered":"2024-05-08T17:17:50.340462+0000","last_unstale":"2024-05-08T17:19:10.890960+0000","last_undegraded":"2024-05-08T17:19:10.890960+0000","last_fullsized":"2024-05-08T17:17:50.339899+0000","mapping_epoch":24,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:04:34.098879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2],"acting":[2],"avail_no_missing":["2"],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":260,"reported_epoch":52,"state":"active+clean","last_fresh":"2024-05-08T17:19:10.890359+0000","last_change":"2024-05-08T17:18:45.806863+0000","last_active":"2024-05-08T17:19:10.890359+0000","last_peered":"2024-05-08T17:19:10.890359+0000","last_clean":"2024-05-08T17:19:10.890359+0000","last_became_active":"2024-05-08T17:18:45.806454+0000","last_became_peered":"2024-05-08T17:18:45.806454+0000","last_unstale":"2024-05-08T17:19:10.890359+0000","last_undegraded":"2024-05-08T17:19:10.890359+0000","last_fullsized":"2024-05-08T17:19:10.890359+0000","mapping_epoch":42,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":43,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:56:10.642935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":14,"acting":14,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1057856,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542181,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":198048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542179,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":198048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542179,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343824,"statfs":{"total":96636764160,"available":96608075776,"internally_reserved":0,"allocated":1019904,"data_stored":726976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[0,1,2,3,4,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":13,"seq":55834574883,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":198048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":13,"seq":55834574861,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27204,"kb_used_data":184,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344636,"statfs":{"total":96636764160,"available":96608907264,"internally_reserved":0,"allocated":188416,"data_stored":67460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574883,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":198048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574882,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27496,"kb_used_data":476,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344344,"statfs":{"total":96636764160,"available":96608608256,"internally_reserved":0,"allocated":487424,"data_stored":198048,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:19:15.272 INFO:tasks.thrashosds.thrasher:Setting 2.3 to [4, 5] 2024-05-08T17:19:15.272 INFO:tasks.thrashosds.thrasher:cmd ['osd', 'pg-upmap', '2.3', '4', '5'] 2024-05-08T17:19:15.272 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pg-upmap 2.3 4 5 2024-05-08T17:19:15.281 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:15.282 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:15.282 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:15.282 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:15.291 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:15.297 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:15.297 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:15.317 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:15.317 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:15.330 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:15.333+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:15.334 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:15.335 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:15.353 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:15.353 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:15.372 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:15.372 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:15.372 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:15.393 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:15.394 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:15.408 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:15.408 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:15.411 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:15.411 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:15.415 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:15.416 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:15.418 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:15.459 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:15.462+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:15.532 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:15.536+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:15.564 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:15.564 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:15.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:15.633 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:15.636+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:15.719 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:15.719 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:15.728 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:15.771 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:15.775+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:15.833 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:15.833 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:15.835 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:15.839+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:15.842 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:15.936 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:15.940+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:15.989 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:15.990 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:15.999 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:16.044 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:16.047+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:16.116 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.116 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.125 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:16.166 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:16.169+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:16.240 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:16.243+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:16.271 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.272 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.280 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:16.341 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:16.344+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:16.425 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.425 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.434 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:16.441 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:16.445+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:16.459 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:16.538 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.539 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.543 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:16.546+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:16.547 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:16.644 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:16.647+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:16.694 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.694 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.702 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:16.745 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:16.748+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:16.816 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":5,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:16.846 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:16.849+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:16.847 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:16.848 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:16.857 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:16.946 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:16.949+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:17.002 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:17.003 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:17.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:17.048 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:17.051+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:17.149 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:17.153+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:17.156 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:17.157 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:17.165 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:17.250 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:17.253+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:17.271 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:17.271 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:17.280 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:17.351 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:17.355+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:17.390 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:17.391 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:17.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:17.417 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:17.452 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:17.455+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:17.551 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:17.551 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:17.554 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:17.556+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:17.561 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:17.567 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:17.567 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:17.585 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:17.586 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:17.602 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:17.603 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:17.619 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:17.619 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:17.636 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:17.637 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:17.637 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:17.655 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:17.655 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:17.655 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:17.658+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:17.671 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:17.671 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:17.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.86790905500001, 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:17.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:17.677 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.845935903, 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:17.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:17.681 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:17.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.845479547, 2024-05-08T17:19:17.683 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.684 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:17.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.686 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.687 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.82643779099999, 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.688 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.689 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.690 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:17.691 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.818148732, 2024-05-08T17:19:17.692 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.693 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.694 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.695 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.696 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.81766072799999, 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:17.697 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:17.698 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.699 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.700 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.817186295, 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.701 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.702 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.703 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:17.704 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:17.705 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.81679292000001, 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:17.706 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:17.707 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.708 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.709 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.81060648299999, 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.710 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.711 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:17.712 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:17.713 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.805803032, 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.714 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:17.715 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.716 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:17.717 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.803227574, 2024-05-08T17:19:17.718 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.719 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.720 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:17.721 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.80044187799999, 2024-05-08T17:19:17.722 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.723 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.724 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.725 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.726 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:17.727 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.79543925499999, 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.728 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:17.729 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.730 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.731 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.62426319400001, 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.732 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.733 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:17.734 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:17.735 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 163.213679224, 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.736 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.737 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.738 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.739 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:17.740 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 162.740882113, 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:17.741 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:17.742 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.743 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.744 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 162.74029654700001, 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:17.745 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.746 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.747 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:17.748 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.749 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 162.73985714, 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.750 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.751 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:17.752 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.753 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:17.754 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 162.739209155, 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:17.755 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.756 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:17.757 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:17.758 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.759 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 162.73374742799999, 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.760 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.761 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:17.762 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:17.763 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:17.764 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:17.765 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:17.765 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:17.765 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:17.766 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:17.767 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:17.770+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:17.869 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:17.872+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:17.911 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:17.912 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:17.912 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:17.912 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:17.920 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:17.969 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:17.972+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:18.026 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.026 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:18.027 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:18.027 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:18.027 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.035 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:18.070 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:18.073+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:18.170 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:18.173+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.182 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 89.903720203000006, 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:18.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:18.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 27.841716308999999, 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:18.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.192 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:18.193 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.195 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:18.278 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:18.281+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:18.341 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.341 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:18.341 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:18.341 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:18.391 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:18.393+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:18.473 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:18.476+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:18.495 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.496 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:18.496 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:18.496 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:18.496 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.504 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:18.588 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:18.591+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:18.651 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.651 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:18.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 90.372913503000007, 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.652 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:18.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:18.655 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:18.702 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:18.704+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:18.776 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:18.779+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.807 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:18.807 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:18.807 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.816 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:18.878 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:18.881+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:18.921 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:18.921 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:18.921 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:18.930 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:18.980 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:18.982+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:19.078 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:19.078 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:19.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:19.078 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:19.078 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:19.080 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:19.083+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:19.087 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:19.182 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:19.185+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:19.203 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:19.203 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:19.204 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.213 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:19.283 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:19.286+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:19.319 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.319 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:19.320 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:19.320 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.328 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:19.384 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:19.387+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:19.432 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.432 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:19.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:19.432 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:19.432 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:19.487 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:19.490+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:19.546 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:19.546 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:19.546 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.554 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:19.586 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:19.589+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:19.687 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:19.690+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:19.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:19.785 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:19.785 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:19.785 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.785 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:19.826 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:19.829+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:19.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:19.889 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:19.892+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:19.930 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:19.930 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:19.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:19.930 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:19.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.738364040999997, 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:19.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:19.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 45.735328523, 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:19.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:19.939 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:19.939 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:19.939 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:19.939 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:19.940 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:20.024 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:20.027+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:20.085 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:20.085 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:20.085 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:20.090 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:20.093+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:20.094 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:20.184 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":40,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":8,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:20.193 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:20.194+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:20.239 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:20.239 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:20.239 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:20.239 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:20.240 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:20.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:20.292 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:20.295+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:20.354 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.55406442399999, 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.355 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.356 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.357 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:20.358 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:20.359 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:20.360 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.52110083299999, 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:20.361 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.362 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.363 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.364 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.365 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.366 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.503336418, 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.367 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:20.368 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:20.369 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:20.370 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:20.371 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.50324338799999, 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.372 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.373 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:20.374 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:20.375 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:20.376 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.377 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.475145363, 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.378 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.379 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.380 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.381 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.382 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.47503528600001, 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.383 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.384 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.385 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.386 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.387 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.388 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.45117475800001, 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.389 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:20.390 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.391 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:20.392 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.393 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.30981514999999, 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.394 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.395 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.396 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.397 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.398 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.399 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.30969795499999, 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.400 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.401 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:20.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:20.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 166.30967300399999, 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.405 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91180214600001, 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91178656599999, 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.416 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:20.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91176724600001, 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91169074300001, 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.427 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:20.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:20.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.432 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91166914799999, 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.433 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:20.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:20.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.91164275400001, 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.438 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.439 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:20.443 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.911568528, 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:20.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:20.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.448 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.425775031, 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.449 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:20.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.425761744, 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.454 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:20.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:20.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:20.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:20.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 165.42569267600001, 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:20.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:20.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:20.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:20.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:20.472 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:20.474 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:20.475 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:20.476+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:20.575 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:20.577+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:20.579 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:20.579 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:20.579 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:20.580 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:20.588 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:20.676 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:20.679+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:20.735 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:20.735 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:20.735 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:20.735 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:20.735 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:20.744 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:20.777 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:20.780+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:20.878 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:20.881+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:20.892 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:20.892 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:20.892 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.08963196100001, 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:20.893 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.894 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.895 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.896 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.06765880899999, 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.897 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:20.898 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:20.899 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.900 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.06720245299999, 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:20.901 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.902 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.903 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:20.904 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:20.905 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.04816069699999, 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.906 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.907 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.908 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.909 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.03987163799999, 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:20.910 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:20.911 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:20.912 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:20.913 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.03938363399999, 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.914 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.915 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.916 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.917 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.038909201, 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.918 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:20.919 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:20.920 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:20.921 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.03851582600001, 2024-05-08T17:19:20.922 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.923 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:20.924 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:20.925 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.926 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.03232938900001, 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.027525938, 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:20.932 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:20.933 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:20.934 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.02495048, 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.935 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.936 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.937 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.938 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.02216478400001, 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.939 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:20.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:20.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 167.01716216099999, 2024-05-08T17:19:20.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.944 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.945 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:20.946 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.947 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 166.8459861, 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.948 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.949 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.950 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 166.43540213, 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.955 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 165.96260501899999, 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.956 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:20.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 165.96201945300001, 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.960 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:20.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:20.962 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:20.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 165.96158004599999, 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:20.964 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.967 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:20.968 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 165.96093206099999, 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:20.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:20.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:20.972 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 165.95547033400001, 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:20.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:20.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:20.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.976 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:20.977 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:20.977 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:20.978 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:20.979 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:20.981+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:21.081 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:21.083+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:21.084 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.085 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:21.085 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:21.085 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:21.182 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:21.184+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:21.200 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.200 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:21.200 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:21.200 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:21.201 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:21.293 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:21.296+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.356 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 93.077699859999996, 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:21.357 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.358 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.359 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.360 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.361 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 31.015695965999999, 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:21.362 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:21.363 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:21.364 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:21.365 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.366 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.366 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:21.366 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.367 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:21.408 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:21.410+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:21.487 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:21.489+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:21.512 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:21.512 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:21.512 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.521 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:21.587 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:21.590+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:21.666 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.666 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:21.666 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:21.666 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:21.666 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:21.691 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:21.693+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:21.782 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.782 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 93.504062383999994, 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:21.783 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.784 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.785 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.786 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:21.787 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.787 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:21.787 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:21.787 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:21.792 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:21.793+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:21.892 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:21.894+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:21.939 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:21.939 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:21.939 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:21.939 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:21.947 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:22.032 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:22.035+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:22.093 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:22.096+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:22.094 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:22.094 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:22.094 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:22.094 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:22.094 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:22.103 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:22.194 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:22.197+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:22.248 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:22.249 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:22.249 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:22.249 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:22.249 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:22.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:22.295 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:22.298+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:22.374 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:22.375 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:22.375 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:22.384 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:22.428 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:22.431+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:22.488 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:22.488 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:22.489 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:22.497 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:22.497 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:22.500+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:22.598 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:22.601+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:22.642 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:22.642 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:22.643 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:22.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:22.699 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:22.701+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:22.756 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:22.756 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:22.757 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:22.765 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:22.809 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:22.812+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:22.870 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:22.871 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:22.871 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:22.880 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:22.901 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:22.904+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:23.002 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:23.005+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.834356894999999, 2024-05-08T17:19:23.026 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.027 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:23.028 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:23.029 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:23.030 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 48.831321377000002, 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.031 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.032 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.033 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.034 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.035 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:23.036 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:23.120 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:23.123+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:23.182 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:23.182 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:23.183 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:23.183 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:23.191 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:23.198 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:23.243 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:23.246+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:23.305 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:23.307+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:23.337 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:23.338 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:23.338 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:23.346 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:23.406 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:23.409+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:23.452 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.65198828499999, 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.453 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:23.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:23.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:23.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.61902469399999, 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.460 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:23.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:23.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:23.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.601260279, 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.465 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.470 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.60116724900001, 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:23.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:23.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:23.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.57306922399999, 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.476 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.57295914700001, 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:23.481 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.54909861900001, 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:23.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:23.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:23.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.492 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.40773901099999, 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.40762181599999, 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.498 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:23.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:23.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.40759686499999, 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.504 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:23.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:23.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:23.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00972600700001, 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:23.509 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00971042699999, 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:23.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:23.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00969110700001, 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.520 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:23.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:23.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:23.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00961460400001, 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:23.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:23.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:23.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:23.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:23.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00959300900001, 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.534 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:23.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.00956661500001, 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 169.009492389, 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.545 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 168.523698892, 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 168.523685605, 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.556 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:23.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 168.52361653700001, 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.562 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:23.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:23.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:23.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:23.567 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:23.568 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:23.569 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:23.571+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:23.669 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:23.672+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:23.676 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:23.676 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:23.676 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:23.684 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:23.770 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:23.772+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:23.790 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:23.790 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:23.790 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:23.790 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:23.790 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:23.799 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:23.882 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:23.885+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:23.949 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.14453848599999, 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:23.950 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.951 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.952 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:23.953 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.954 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:23.955 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.122565334, 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.956 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:23.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:23.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:23.960 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.122108978, 2024-05-08T17:19:23.961 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:23.962 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.962 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.962 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.962 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.962 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.963 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.964 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.965 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:23.966 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.10306722199999, 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.967 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:23.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:23.971 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.094778163, 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:23.972 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.973 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:23.974 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.975 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:23.976 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.977 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.094290159, 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.978 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:23.983 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.093815726, 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.984 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:23.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.09342235099999, 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:23.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:23.994 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.08723591399999, 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:23.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:23.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:23.998 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:23.999 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.082432463, 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.000 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:24.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:24.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:24.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:24.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:24.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:24.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.07985700500001, 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:24.010 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:24.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:24.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:24.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.07707130899999, 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.019 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:24.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 170.07206868599999, 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:24.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:24.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.90089262500001, 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.030 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:24.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.49030865500001, 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.035 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:24.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:24.039 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.017511544, 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.040 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:24.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.045 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.01692597799999, 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.016486571, 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:24.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:24.054 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.015838586, 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.055 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:24.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:24.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 169.01037685899999, 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:24.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:24.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.064 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:24.067 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:24.068+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:24.166 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:24.169+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:24.214 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.214 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:24.215 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:24.215 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.224 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:24.267 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:24.270+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:24.329 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.329 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:24.329 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:24.329 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:24.329 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.338 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:24.383 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:24.385+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:24.444 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.444 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:24.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:24.444 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 96.165885433, 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:24.445 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:24.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:24.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:24.450 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 34.103881539, 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:24.451 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:24.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.455 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.456 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.456 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.456 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.457 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:24.499 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:24.501+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:24.570 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:24.572+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:24.604 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.605 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:24.605 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:24.605 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.613 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:24.671 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:24.673+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:24.760 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.760 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:24.761 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:24.761 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:24.761 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.769 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:24.771 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:24.773+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:24.871 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:24.874+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.875 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 96.596866105999993, 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.876 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.877 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:24.878 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:24.879 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:24.880 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:24.884 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:24.975 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:24.977+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:25.031 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:25.031 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:25.031 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:25.031 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:25.040 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:25.081 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:25.084+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:25.178 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:25.181+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:25.186 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:25.186 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:25.186 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:25.194 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:25.279 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:25.282+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:25.340 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:25.340 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:25.340 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:25.340 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:25.340 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:25.349 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:25.393 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:25.396+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:25.466 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:25.466 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:25.466 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:25.467 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:25.476 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:25.481 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:25.483+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:25.582 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:25.584+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:25.621 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:25.621 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:25.621 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:25.621 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:25.621 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:25.629 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:25.682 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:25.685+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:25.734 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:25.735 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:25.735 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:25.735 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:25.735 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:25.743 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:25.783 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:25.786+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:25.884 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:25.887+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:25.887 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:25.887 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:25.887 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:25.896 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:25.985 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:25.987+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:26.001 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:26.002 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:26.002 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:26.002 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:26.002 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:26.011 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:26.096 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:26.099+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:26.158 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.966692969999997, 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:26.159 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.160 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.161 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.162 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.163 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:26.164 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 51.963657452, 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:26.165 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.166 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:26.167 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:26.168 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:26.169 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.170 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.171 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:26.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:26.214 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:26.216+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:26.288 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:26.291+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:26.318 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:26.318 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:26.319 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:26.319 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:26.327 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:26.390 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:26.392+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:26.471 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:26.471 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:26.472 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:26.472 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:26.480 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:26.524 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:26.527+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.786310007, 2024-05-08T17:19:26.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:26.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:26.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.753346416, 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.73558200100001, 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.599 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:26.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:26.604 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.735488971, 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.605 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:26.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.707390946, 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.612 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.70728086899999, 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.618 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.68342034099999, 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:26.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:26.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:26.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.542060733, 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.541943538, 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:26.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.541918587, 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:26.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:26.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.14404772899999, 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.645 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:26.646 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:26.647 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:26.648 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.649 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.144032149, 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.650 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.651 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:26.652 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:26.653 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:26.654 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.655 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.14401282899999, 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.656 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:26.657 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:26.658 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.659 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:26.660 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.14393632599999, 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.661 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.662 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.663 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.664 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.665 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.666 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.143914731, 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:26.667 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:26.668 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:26.669 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.670 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.671 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.14388833699999, 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.672 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:26.673 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.674 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:26.675 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:26.676 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.677 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 172.14381411100001, 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:26.678 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:26.679 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:26.680 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:26.681 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:26.682 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 171.65802061400001, 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:26.683 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:26.684 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.685 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.686 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.687 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 171.65800732700001, 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.688 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.689 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.690 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.691 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.692 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:26.693 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 171.65793825899999, 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:26.694 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:26.695 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:26.696 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.697 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.698 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.699 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:26.699 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:26.699 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:26.700 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:26.703+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:26.701 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:26.801 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:26.804+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:26.818 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:26.818 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:26.818 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:26.818 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:26.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:26.901 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:26.904+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:26.935 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:26.935 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:26.935 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:26.935 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:26.935 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:26.944 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:27.002 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:27.005+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:27.091 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.091 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.28867325499999, 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.092 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.093 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.094 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:27.095 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:27.096 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.26670010300001, 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.097 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:27.098 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:27.099 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.100 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.266243747, 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.101 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:27.102 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:27.103 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:27.104 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:27.105 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.247201991, 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.106 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.107 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:27.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.23891293200001, 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:27.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:27.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:27.115 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.238424928, 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.116 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:27.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:27.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:27.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.23795049500001, 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:27.120 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:27.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:27.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.124 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.23755711999999, 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.23137068299999, 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.129 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:27.133 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.22656723200001, 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:27.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.22399177400001, 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:27.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:27.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:27.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:27.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.22120607799999, 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.216203455, 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:27.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:27.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 173.04502739399999, 2024-05-08T17:19:27.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:27.154 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.63444342400001, 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:27.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:27.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:27.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.16164631300001, 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.162 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.16106074699999, 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:27.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:27.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:27.166 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.16062134000001, 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.15997335500001, 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.171 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:27.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 172.15451162799999, 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.175 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:27.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:27.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.179 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:27.181 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:27.183+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:27.281 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:27.284+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:27.287 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:27.287 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:27.287 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.296 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:27.382 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:27.385+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:27.443 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:27.444 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:27.444 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.452 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:27.483 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:27.487+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:27.584 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:27.586+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:27.598 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 99.319882265999993, 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.603 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 37.257878372, 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:27.604 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.607 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.608 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.608 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.608 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.608 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.609 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:27.685 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:27.688+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:27.756 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.756 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:27.757 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:27.757 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.765 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:27.786 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:27.789+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:27.871 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.871 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:27.871 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:27.871 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:27.871 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.880 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:27.923 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:27.926+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 99.707288458999997, 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:27.986 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:27.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:27.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:27.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:27.990 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:27.991 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:27.994+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:27.994 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:28.065 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:28.065 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:28.083 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:28.083 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:28.092 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:28.095+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:28.099 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:28.100 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:28.116 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:28.117 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:28.135 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:28.135 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:28.135 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:28.157 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.157 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.169 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:28.170 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:28.170 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:28.188 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:28.188 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:28.191 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:28.192 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:28.193 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:28.196+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:28.276 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.276 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.285 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:28.328 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:28.331+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:28.392 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.392 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.395 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:28.398+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:28.401 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:28.495 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:28.498+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:28.519 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.519 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.529 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:28.596 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:28.599+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:28.674 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.674 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.683 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:28.697 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:28.699+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:28.788 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.788 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.797 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:28.798 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:28.801+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:28.899 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:28.902+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:28.902 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:28.911 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:29.000 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:29.002+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:29.016 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.016 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.025 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:29.047 INFO:teuthology.orchestra.run.smithi154.stderr:set 2.3 pg_upmap mapping to [4,5] 2024-05-08T17:19:29.047 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:19:29.101 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:29.104+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:29.131 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.132 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:29.224 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:29.227+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:29.285 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.285 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.294 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:29.302 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:29.306+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:29.404 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:29.406+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:29.439 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.439 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.448 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:29.504 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:29.507+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:29.551 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.551 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.560 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:29.565 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":42,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":1,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:29.579 INFO:tasks.mon_recovery.ceph_manager:quorum is size 2 2024-05-08T17:19:29.579 INFO:tasks.mon_recovery:causing some monitor log activity 2024-05-08T17:19:29.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '1 of 30' 2024-05-08T17:19:29.605 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:29.608+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:29.668 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.668 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.679 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:29.706 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:29.709+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:29.789 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.789 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.799 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:29.842 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:29.845+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:29.906 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:29.906 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:29.908 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:29.911+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:29.915 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:30.009 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:30.012+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:30.063 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:30.064 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:30.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '2 of 30' 2024-05-08T17:19:30.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:30.110 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:30.113+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:30.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:30.223 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:30.223 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:30.223 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:30.225+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:30.232 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:30.316 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:30.319+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:30.336 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:30.337 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:30.355 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:30.355 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:30.372 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:30.372 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:30.389 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:30.389 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.395 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 102.116755059, 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.396 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.397 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:30.398 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:30.398 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:30.398 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.398 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.398 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:30.399 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.400 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.401 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:30.402 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 40.054751164999999, 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:30.403 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.404 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.405 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:30.406 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:30.407 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:30.412 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:30.412 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:30.412 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:30.414 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:30.416+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:30.426 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:30.427 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:30.444 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:30.445 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:30.448 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:30.448 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:30.514 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:30.517+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:30.532 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:30.532 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:30.532 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:30.532 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:30.541 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:30.625 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:30.628+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:30.687 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:30.688 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:30.688 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:30.688 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:30.688 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:30.697 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:30.716 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:30.719+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:30.818 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:30.820+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 102.564525746, 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:30.843 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.844 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:30.845 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.846 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:30.847 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:30.852 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:30.919 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:30.922+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:30.958 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:30.959 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:30.959 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:30.959 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:30.968 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:31.019 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:31.022+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:31.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:19:31.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '3 of 30' 2024-05-08T17:19:31.077 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:31.077 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:31.077 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:31.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:31.127 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:31.130+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:31.221 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:31.223+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:31.237 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:31.237 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:31.237 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:31.237 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:31.237 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:31.247 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:31.322 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:31.324+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:31.364 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:31.364 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:31.364 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:31.373 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:31.423 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:31.426+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:31.478 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:31.479 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:31.479 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:31.487 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:31.524 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:31.527+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:31.625 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:31.628+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:31.631 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:31.632 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:31.632 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:31.640 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:31.726 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:31.729+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:31.745 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:31.745 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:31.745 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:31.754 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:31.827 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:31.830+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:31.859 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:31.859 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:31.859 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:31.859 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:31.859 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:31.867 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:31.928 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:31.931+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.820970735000003, 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.013 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.014 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.015 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.016 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:32.017 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 57.817935216999999, 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.018 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.019 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:32.020 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.021 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:32.022 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:32.029 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:32.032+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:32.063 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:19:32.075 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '4 of 30' 2024-05-08T17:19:32.129 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:32.129 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:32.129 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:32.129 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:32.129 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:32.132+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:32.137 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:32.230 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:32.232+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:32.284 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:32.284 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:32.284 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:32.284 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:32.284 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:32.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:32.330 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:32.333+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:32.400 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:32.400 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:32.400 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:32.400 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:32.400 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.599348115, 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.401 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:32.402 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:32.403 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:32.404 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:32.405 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.406 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.566384524, 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.407 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.408 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.409 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.410 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.411 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.54862010900001, 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:32.412 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.413 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:32.414 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:32.415 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.416 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:32.417 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.548527079, 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.418 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.419 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.420 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:32.421 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.422 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.520429054, 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.423 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:32.424 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.425 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.426 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.427 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.52031897699999, 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.428 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.429 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.430 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.431 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.432 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.49645844899999, 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.433 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.434 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.435 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.436 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.437 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.438 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.355098841, 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:32.439 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:32.440 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.441 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.442 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.443 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.354981646, 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.444 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:32.445 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.446 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.447 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.448 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 178.354956695, 2024-05-08T17:19:32.449 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.450 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:32.451 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.452 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:32.453 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.454 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.95708583699999, 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.455 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.456 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.457 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.458 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:32.459 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.957070257, 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.460 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.461 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:32.462 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.463 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.464 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.95705093699999, 2024-05-08T17:19:32.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.469 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.470 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.95697443399999, 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.473 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:32.474 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:32.475 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.956952839, 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.476 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.477 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:32.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:32.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:32.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:32.481 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.95692644499999, 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:32.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:32.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:32.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.95685221900001, 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.487 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:32.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.492 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.47105872200001, 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.493 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:32.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:32.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:32.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.498 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.47104543500001, 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:32.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:32.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:32.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 177.47097636699999, 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:32.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:32.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:32.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:32.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:32.510 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:32.511 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:32.512 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:32.514+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:32.512 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:32.612 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:32.615+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:32.622 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:32.622 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:32.623 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:32.623 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:32.632 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:32.638 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:32.638 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:32.657 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:32.658 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:32.675 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:32.676 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:32.692 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:32.692 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:32.710 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:32.711 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:32.711 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:32.713 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:32.715+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:32.728 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:32.728 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:32.742 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:32.742 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:32.746 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:32.747 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:32.750 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:32.750 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:32.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:32.835 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:32.838+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:32.898 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:32.898 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:32.908 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:32.915 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:32.918+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:33.016 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:33.018+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:33.054 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.054 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.064 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:33.074 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '5 of 30' 2024-05-08T17:19:33.117 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:33.119+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:33.217 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:33.220+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:33.224 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.224 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.234 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:33.318 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:33.321+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:33.381 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.381 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.390 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:33.419 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:33.422+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:33.497 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.497 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.506 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:33.550 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:33.553+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:33.612 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.612 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.620 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:33.621 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:33.623+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:33.721 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:33.723+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:33.766 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.767 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.776 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:33.859 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:33.862+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:33.922 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:33.922 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:33.922 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:33.925+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:33.931 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:34.023 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:34.025+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:34.064 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:34.065 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:34.075 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '6 of 30' 2024-05-08T17:19:34.077 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.077 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:19:34.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:34.129 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:34.131+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:34.224 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:34.227+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:34.240 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.241 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.250 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:34.325 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:34.328+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:34.368 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.368 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.377 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:34.461 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:34.464+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:34.523 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.523 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.527 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:34.531+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:34.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:34.628 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:34.631+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:34.677 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.677 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.685 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:34.751 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:34.769 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:34.772+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:34.829 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:34.832+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:34.834 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:34.835 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:34.844 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:34.875 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:34.876 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:34.893 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:34.893 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:34.909 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:34.910 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:34.926 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:34.926 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:34.930 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:34.933+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:34.944 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:34.944 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:34.944 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:34.962 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:34.963 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:34.981 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:34.981 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:34.984 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:34.984 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:34.993 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:34.993 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:34.993 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:34.993 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:34.993 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:35.002 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:35.045 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:35.048+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:35.064 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:19:35.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '7 of 30' 2024-05-08T17:19:35.108 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:35.108 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.916911794000001, 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:35.109 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:35.110 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.111 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.112 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 60.913876276000003, 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:35.113 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:35.114 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:35.115 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.116 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:35.117 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:35.159 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:35.162+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:35.233 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:35.236+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:35.264 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:35.264 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:35.264 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:35.264 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:35.272 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:35.334 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:35.336+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:35.417 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:35.417 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:35.418 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:35.418 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:35.427 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:35.471 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:35.474+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:35.534 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:35.534 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:35.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:35.534 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:35.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.73386905800001, 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:35.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:35.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:35.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.540 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.70090546700001, 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:35.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:35.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.683141052, 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.546 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.68304802200001, 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:35.551 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:35.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:35.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.556 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.65494999699999, 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:35.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.65483992, 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:35.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:35.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.630979392, 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.567 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:35.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.48961978400001, 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.572 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:35.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:35.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:35.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.577 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.48950258900001, 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:35.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:35.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.48947763800001, 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.583 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.585 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.09160678000001, 2024-05-08T17:19:35.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.591 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:35.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.09159120000001, 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:35.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.09157188, 2024-05-08T17:19:35.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:35.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:35.603 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.603 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.603 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.605 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.606 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.091495377, 2024-05-08T17:19:35.607 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.608 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:35.609 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:35.610 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.611 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.09147378200001, 2024-05-08T17:19:35.612 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.613 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:35.614 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.615 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.616 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.617 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.09144738800001, 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.618 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.619 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.620 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.621 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.622 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 181.091373162, 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.623 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.624 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.625 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.626 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:35.627 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 180.60557966499999, 2024-05-08T17:19:35.628 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.629 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.630 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.631 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.632 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.633 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 180.60556637799999, 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.634 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.635 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.636 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.637 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.638 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 180.60549731, 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:35.639 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.640 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.641 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.642 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:35.643 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:35.644 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:35.646 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:35.646 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:35.649+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:35.746 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:35.749+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:35.752 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:35.752 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:35.752 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:35.752 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:35.761 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:35.847 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:35.851+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:35.866 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:35.866 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:35.866 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:35.875 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:35.949 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:35.952+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:35.982 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:35.982 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:35.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:35.982 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.179507766, 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:35.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:35.985 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.15753461400001, 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:35.987 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:35.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.989 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.15707825800001, 2024-05-08T17:19:35.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.993 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.994 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.13803650200001, 2024-05-08T17:19:35.995 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.996 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.997 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:35.998 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:35.999 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.12974744300001, 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.000 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:36.001 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:36.002 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.003 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.12925943900001, 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.004 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.12878500599999, 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:36.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:36.015 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.128391631, 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:36.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:36.019 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.122205194, 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:36.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:36.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:36.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.11740174299999, 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.11482628499999, 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:36.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:36.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.031 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.112040589, 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:36.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:36.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 182.10703796600001, 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:36.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.935861905, 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:36.040 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.52527793499999, 2024-05-08T17:19:36.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:36.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:36.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.048 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.05248082400001, 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:36.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.052 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.051895258, 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:36.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:36.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:36.056 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.05145585099999, 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.05080786600001, 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.061 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:36.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 181.045346139, 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.065 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.066 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:36.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:36.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.069 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.071 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:36.071 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:36.074+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:36.080 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '8 of 30' 2024-05-08T17:19:36.171 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:36.174+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:36.230 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.230 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:36.230 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:36.230 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.241 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:36.271 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:36.275+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:36.372 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:36.376+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:36.388 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.389 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:36.389 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:36.389 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:36.389 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:36.474 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:36.476+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.505 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 108.22696081700001, 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:36.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:36.510 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 46.164956922999998, 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:36.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:36.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.514 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.515 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:36.575 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:36.578+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:36.621 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.621 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:36.622 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:36.622 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:36.676 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:36.679+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:36.776 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:36.780+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:36.782 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.782 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:36.782 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:36.782 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:36.782 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.791 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:36.877 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:36.881+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 108.660375493, 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:36.939 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:36.940 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.941 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:36.942 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:36.943 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:36.948 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:36.985 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:36.990 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:36.993+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:37.079 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:37.083+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:37.081 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:19:37.085 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '9 of 30' 2024-05-08T17:19:37.102 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:37.102 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:37.102 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:37.102 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:37.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:37.117 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:37.117 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:37.137 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:37.137 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:37.156 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:37.156 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:37.175 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:37.175 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:37.180 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:37.184+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:37.194 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:37.194 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:37.195 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:37.218 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:37.218 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:37.236 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:37.236 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:37.237 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.237 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.241 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:37.241 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:37.250 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:37.291 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:37.294+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:37.382 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:37.386+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:37.397 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.397 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.406 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:37.484 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:37.487+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:37.523 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.523 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.532 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:37.585 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:37.588+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:37.638 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.638 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.646 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:37.687 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:37.690+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:37.787 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:37.790+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:37.791 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.791 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.799 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:37.888 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:37.891+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:37.905 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:37.905 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:37.913 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:37.989 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:37.992+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:38.017 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.018 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.026 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:38.076 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:19:38.088 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '10 of 30' 2024-05-08T17:19:38.092 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:38.095+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:38.171 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.171 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.180 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:38.191 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:38.194+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:38.284 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.285 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.292 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:38.295+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:38.293 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:38.393 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:38.396+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:38.438 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.438 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.447 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:38.531 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:38.534+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:38.592 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.593 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.595 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:38.598+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:38.601 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:38.696 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:38.699+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:38.706 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.707 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.716 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:38.799 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:38.803+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:38.860 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:38.860 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:38.869 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:38.911 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:38.914+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:39.000 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:39.003+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:39.017 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:39.017 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:39.027 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:39.065 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:39.066 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:39.066 INFO:tasks.thrashosds.thrasher:inject_pause on osd.1 2024-05-08T17:19:39.066 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:19:39.066 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:19:39.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok config set filestore_inject_stall 3 2024-05-08T17:19:39.088 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '11 of 30' 2024-05-08T17:19:39.100 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:39.104+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:39.176 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:39.176 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "" 2024-05-08T17:19:39.176 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:39.177 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:39.177 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:39.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:39.229 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:39.232+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:39.243 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:39.314 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:39.318+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:39.340 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:39.340 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:39.350 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:39.396 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:39.397 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:39.403 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:39.406+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:39.416 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:39.416 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:39.434 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:39.435 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:39.452 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:39.452 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:39.470 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:39.470 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:39.470 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:39.487 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:39.488 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 111.222702731, 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:39.501 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:39.506 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 49.160698836999998, 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:39.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:39.510 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:39.511 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:39.511 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:39.512 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:39.512 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:39.513 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:39.513 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:39.514+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:39.612 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:39.616+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:39.659 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:39.660 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:39.660 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:39.660 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:39.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:39.713 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:39.716+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:39.814 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:39.817+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:39.814 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:39.814 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:39.814 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:39.814 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:39.815 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:39.823 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:39.915 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:39.918+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:39.928 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:39.928 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:39.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:39.928 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:39.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 111.650097558, 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:39.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:39.930 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:39.931 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:39.932 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:39.937 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:40.016 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:40.019+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:40.043 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:40.043 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:40.043 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:40.043 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:40.052 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:40.088 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '12 of 30' 2024-05-08T17:19:40.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:19:40.130 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:40.133+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:40.161 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:40.161 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:40.161 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:40.161 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:40.161 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:40.171 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:40.257 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:40.260+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:40.319 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:40.323+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:40.323 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:40.323 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:40.323 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:40.323 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:40.323 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:40.332 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:40.420 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:40.424+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:40.449 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:40.449 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:40.449 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:40.449 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:40.458 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:40.521 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:40.524+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:40.563 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:40.563 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:40.563 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:40.563 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:40.563 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:40.573 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:40.621 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:40.625+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:40.677 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:40.677 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:40.677 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:40.677 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:40.678 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:40.686 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:40.722 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:40.725+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:40.792 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:40.792 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:40.792 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:40.801 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:40.844 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:40.847+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:40.905 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:40.905 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:40.905 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:40.905 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:40.905 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:40.914 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:40.955 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:40.958+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:41.025 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:41.028+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:41.059 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:41.059 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.867484738000002, 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.060 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.061 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:41.062 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:41.063 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 66.864449219999997, 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.064 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.065 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.066 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.067 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.068 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.068 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.068 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:41.069 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:41.077 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:19:41.094 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '13 of 30' 2024-05-08T17:19:41.154 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:41.157+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:41.214 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:41.215 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:41.215 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:41.215 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:41.223 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:41.226 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:41.230+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:41.327 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:41.330+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:41.367 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:41.367 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:41.367 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:41.367 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:41.367 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:41.375 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:41.428 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:41.431+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.681184046, 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.482 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.488 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.648220455, 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:41.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:41.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.494 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.63045604000001, 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.495 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:41.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:41.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.499 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.63036301, 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:41.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:41.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.60226498500001, 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.507 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:41.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.512 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.60215490799999, 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:41.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:41.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.57829437999999, 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.436934772, 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.524 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:41.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.436817577, 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.530 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.436792626, 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.536 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:41.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:41.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:41.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03892176799999, 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:41.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:41.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.038906188, 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:41.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.554 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03888686799999, 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:41.560 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03881036499999, 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.561 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:41.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.567 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03878877, 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.568 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:41.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:41.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03876237599999, 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:41.573 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.578 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 187.03868815000001, 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:41.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.584 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 186.55289465300001, 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:41.585 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.586 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.587 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.588 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:41.589 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.590 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 186.55288136600001, 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.591 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.592 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:41.593 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:41.594 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:41.595 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:41.596 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 186.55281229799999, 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:41.597 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:41.598 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.599 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.600 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:41.601 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:41.602 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:41.603 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:41.603 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:41.605 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:41.608+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:41.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:41.706 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:41.709+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:41.714 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:41.714 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:41.714 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:41.714 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:41.724 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:41.729 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:41.730 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:41.748 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:41.748 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:41.766 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:41.766 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:41.799 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:41.800 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:41.807 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:41.811+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:41.831 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:41.831 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:41.831 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:41.849 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:41.849 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:41.860 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:41.860 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:41.866 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:41.866 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:41.871 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:41.871 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:41.872 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:41.908 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:41.911+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:42.010 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:42.013+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:42.015 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.015 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.024 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:42.093 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '14 of 30' 2024-05-08T17:19:42.110 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:42.113+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:42.135 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.135 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.144 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:42.228 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:42.232+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:42.294 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.294 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.304 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:42.312 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:42.316+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:42.414 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:42.417+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:42.453 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.453 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.462 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:42.515 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:42.518+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:42.570 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.570 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.577 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:42.621 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:42.625+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.683 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:42.733 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:42.737+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:42.817 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:42.820+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:42.838 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.838 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:42.847 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:42.931 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:42.935+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:42.993 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:42.993 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.002 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:43.043 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:43.047+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:43.096 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:19:43.097 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '15 of 30' 2024-05-08T17:19:43.120 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:43.123+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:43.151 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.151 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.161 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:43.246 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:43.250+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:43.314 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.315 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.322 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:43.325+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:43.323 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:43.423 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:43.426+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:43.439 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.440 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.449 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:43.524 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:43.527+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:43.555 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.555 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.564 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:43.624 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:43.628+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:43.667 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.667 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.676 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:43.725 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:43.728+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:43.781 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.781 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.789 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:43.826 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:43.829+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:43.873 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:43.927 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:43.930+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:43.933 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:43.933 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:43.942 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:43.996 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:43.996 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:44.013 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:44.013 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:44.028 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:44.031+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:44.029 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:44.029 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:44.046 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:44.046 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:44.063 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:44.064 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:44.064 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:44.080 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:44.080 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:44.088 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:19:44.090 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.899062545999996, 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:44.091 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:44.092 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:44.093 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.094 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 69.896027028000006, 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.095 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.096 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.097 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:44.098 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.099 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.099 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.099 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.099 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.099 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:44.100 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:44.102 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '16 of 30' 2024-05-08T17:19:44.103 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:44.103 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:44.105 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:44.106 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:44.141 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:44.144+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:44.187 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:44.187 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:44.187 INFO:tasks.thrashosds.thrasher:inject_pause on osd.3 2024-05-08T17:19:44.188 INFO:tasks.thrashosds.thrasher:Testing filestore_inject_stall pause injection for duration 3 2024-05-08T17:19:44.188 INFO:tasks.thrashosds.thrasher:Checking after 0, should_be_down=False 2024-05-08T17:19:44.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok config set filestore_inject_stall 3 2024-05-08T17:19:44.232 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:44.236+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:44.246 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:44.246 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:44.246 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:44.247 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:44.255 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:44.297 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:44.297 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "filestore_inject_stall = '' (not observed, change may require restart) " 2024-05-08T17:19:44.297 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:44.347 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:44.351+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:44.360 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:44.360 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:44.360 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:44.360 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:44.361 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:44.369 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:44.432 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:44.435+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.67629641900001, 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:44.477 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.478 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.479 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:44.480 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.481 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:44.482 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.64333282800001, 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:44.483 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.484 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:44.485 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:44.486 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:44.487 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.488 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.625568413, 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.489 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:44.490 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:44.491 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:44.492 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.493 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.62547538300001, 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.494 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:44.495 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.496 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.497 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.498 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.59737735799999, 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.499 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.500 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.501 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.502 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.503 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.597267281, 2024-05-08T17:19:44.504 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.505 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.506 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.507 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.508 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.509 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.573406753, 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.510 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.511 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:44.512 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:44.513 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:44.514 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.43204714500001, 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.515 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:44.516 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.517 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.518 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.519 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.43192995000001, 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.520 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.521 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.522 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.523 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.524 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.43190499900001, 2024-05-08T17:19:44.525 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.526 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.527 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.528 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.529 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.530 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.03403414100001, 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.531 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:44.532 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:44.533 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:44.534 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:44.535 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.03401856100001, 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.536 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.537 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.538 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.539 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.540 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.033999241, 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:44.541 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.542 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.543 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.544 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:44.545 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.546 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.033922738, 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.547 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.548 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.549 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.550 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.551 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.03390114300001, 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.552 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.553 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.554 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.555 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.556 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.03387474900001, 2024-05-08T17:19:44.557 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.558 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.559 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.560 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.561 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:44.562 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 190.033800523, 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:44.563 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.564 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.565 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.566 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.567 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 189.54800702599999, 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.568 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.569 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.570 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.571 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.572 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 189.54799373899999, 2024-05-08T17:19:44.573 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.574 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.575 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.576 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.577 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.578 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 189.547924671, 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.579 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:44.580 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:44.581 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:44.582 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:44.583 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:44.584 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:44.585 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:44.586 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:44.589+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:44.686 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:44.690+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:44.692 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:44.692 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:44.692 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:44.692 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:44.701 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:44.787 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:44.790+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:44.849 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:44.849 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:44.849 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:44.849 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:44.849 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:44.858 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:44.888 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:44.892+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:44.989 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:44.993+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:45.004 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.004 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:45.004 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:45.004 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.20150573699999, 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.005 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.006 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.007 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.008 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.179532585, 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.009 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:45.010 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:45.011 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.012 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.179076229, 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.013 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.014 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.015 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.016 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.160034473, 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.017 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:45.018 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.019 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.020 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.151745414, 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.021 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:45.022 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:45.023 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:45.024 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.15125741, 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.025 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.026 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.027 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.028 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.15078297700001, 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:45.029 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.15038960199999, 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.034 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.14420316499999, 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.13939971400001, 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.042 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.043 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.044 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.045 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.13682425600001, 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.046 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.047 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.048 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.049 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.13403855999999, 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.050 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.051 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.052 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.053 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 191.129035937, 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.054 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.055 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:45.056 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:45.057 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.95785987599999, 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.058 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:45.059 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:45.060 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:45.061 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.54727590600001, 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.062 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.063 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:45.064 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.065 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.074478795, 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.066 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.067 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.068 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:45.069 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.07389322899999, 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.070 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:45.071 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.072 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:45.073 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.073453822, 2024-05-08T17:19:45.074 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.075 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.076 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.077 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.072805837, 2024-05-08T17:19:45.078 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.079 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.080 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.081 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:45.082 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 190.06734410999999, 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.083 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.084 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.085 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.086 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:45.090 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:45.093+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:45.101 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '17 of 30' 2024-05-08T17:19:45.191 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:45.195+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:45.250 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.250 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:45.251 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:45.251 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.260 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:45.303 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:45.307+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:45.368 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.368 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:45.368 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:45.368 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:45.368 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.377 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:45.419 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:45.422+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:45.493 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:45.497+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:45.525 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 117.24661549699999, 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.526 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:45.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 55.184611603, 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:45.532 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.536 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.536 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.537 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:45.594 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:45.597+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:45.683 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:45.683 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:45.683 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.692 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:45.695 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:45.697+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:45.795 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:45.798+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:45.798 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:45.798 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.807 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:45.896 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:45.898+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 117.677307138, 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:45.956 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.957 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:45.958 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.959 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.960 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:45.960 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:45.960 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:45.964 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:46.008 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:46.012+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:46.070 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:46.070 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:46.070 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:46.079 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:46.098 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:46.100+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:46.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '18 of 30' 2024-05-08T17:19:46.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:46.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:19:46.199 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:46.202+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:46.256 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:46.256 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:46.257 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:46.257 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:46.257 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:46.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:46.278 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:46.279 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:46.296 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:46.297 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:46.300 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:46.303+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:46.315 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:46.315 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:46.348 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:46.349 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:46.385 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:46.386 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:46.386 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:46.386 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:46.386 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:46.396 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:46.422 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:46.422 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:46.445 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:46.448+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:46.457 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:46.457 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:46.461 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:46.462 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:46.504 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:46.507+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:46.513 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:46.513 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:46.522 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:46.606 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:46.609+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:46.669 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:46.669 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:46.678 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:46.707 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:46.710+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:46.808 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:46.811+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:46.824 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:46.824 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:46.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:46.920 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:46.923+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:46.979 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:46.979 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:46.988 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:47.009 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:47.013+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:47.090 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:19:47.110 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:47.113+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:47.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '19 of 30' 2024-05-08T17:19:47.135 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.135 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.144 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:47.211 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:47.214+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:47.248 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.249 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.257 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:47.312 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:47.315+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:47.402 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.402 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.411 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:47.412 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:47.416+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:47.514 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:47.517+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:47.515 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.516 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.524 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:47.615 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:47.618+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:47.669 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.669 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.678 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:47.716 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:47.719+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:47.784 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.784 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.793 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:47.836 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:47.840+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:47.898 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:47.898 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:47.907 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:47.917 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:47.921+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:48.019 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:48.022+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:48.053 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:48.053 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:48.062 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:48.105 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '20 of 30' 2024-05-08T17:19:48.120 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:48.123+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:48.171 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:48.171 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:48.181 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:48.222 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:48.226+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:48.321 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:48.325+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:48.330 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:48.331 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:48.340 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:48.423 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:48.427+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:48.463 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:48.490 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:48.490 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:48.500 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:48.525 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:48.527+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:48.589 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:48.590 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:48.607 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:48.608 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:48.625 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:48.625 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:48.625 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:48.628+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:48.643 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:48.643 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:48.650 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:48.650 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:48.650 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:48.650 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:48.660 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:48.661 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:48.662 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:48.662 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:48.682 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:48.683 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:48.700 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:48.701 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:48.704 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:48.704 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:48.746 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:48.749+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:48.770 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:48.771 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:48.771 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:48.771 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:48.771 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:48.779 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:48.862 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:48.866+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:48.925 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 120.646438532, 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:48.926 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:48.927 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:48.928 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:48.929 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:48.930 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:48.934+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:48.934 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:49.031 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:49.035+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:49.080 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:49.080 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:49.080 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:49.081 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:49.089 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:49.107 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '21 of 30' 2024-05-08T17:19:49.112 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:19:49.133 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:49.136+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:49.199 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:49.199 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:49.199 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:49.199 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:49.199 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:49.209 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:49.250 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:49.253+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:49.307 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:49.307 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:49.308 INFO:tasks.thrashosds.thrasher:thrash_pg_upmap 2024-05-08T17:19:49.308 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump -f json-pretty 2024-05-08T17:19:49.351 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:49.355+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:49.360 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:49.361 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:49.361 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:49.361 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:19:49.361 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:49.371 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:49.435 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:49.437+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:49.489 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:49.489 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:49.489 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:49.489 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:49.498 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:49.536 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:49.540+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:49.611 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "epoch": 62, 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "fsid": "c69ba297-505d-4f4e-b390-bd22ef533cfc", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "created": "2024-05-08T17:16:22.335952+0000", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "modified": "2024-05-08T17:19:47.090999+0000", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "last_up_change": "2024-05-08T17:17:49.336612+0000", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "last_in_change": "2024-05-08T17:18:41.921552+0000", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": "sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_num": 5799936, 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_set": [ 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "pglog_hardlimit", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "purged_snapdirs", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "recovery_deletes", 2024-05-08T17:19:49.612 INFO:teuthology.orchestra.run.smithi154.stdout: "sortbitwise" 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_version": 4, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "full_ratio": 0.94999998807907104, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "backfillfull_ratio": 0.89999997615814209, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "nearfull_ratio": 0.85000002384185791, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_snapshot": "", 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_max": 2, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "max_osd": 8, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "require_min_compat_client": "luminous", 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "min_compat_client": "luminous", 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "require_osd_release": "squid", 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "allow_crimson": false, 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: "pools": [ 2024-05-08T17:19:49.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 1, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": ".mgr", 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:32.734319+0000", 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:19:49.614 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 1, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 1, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 1, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 1, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 1, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:19:49.615 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "16", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:19:49.616 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:19:49.617 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:19:49.618 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "options": { 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_max": 32, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_min": 1 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:19:49.619 INFO:teuthology.orchestra.run.smithi154.stdout: "mgr": {} 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 6.9000000953674316, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 6.9000000953674316, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 0.28999999165534973, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.620 INFO:teuthology.orchestra.run.smithi154.stdout: "pool": 2, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_name": "rbd", 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "create_time": "2024-05-08T17:16:33.268021+0000", 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "flags": 1, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "flags_names": "hashpspool", 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "type": 1, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 2, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "min_size": 1, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_rule": 0, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_count": 0, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_target": 0, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_barrier": 0, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "peering_crush_bucket_mandatory_member": 2147483647, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "object_hash": 2, 2024-05-08T17:19:49.621 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_autoscale_mode": "off", 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num": 8, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num": 8, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_placement_num_target": 8, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_target": 8, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_num_pending": 8, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "last_pg_merge_meta": { 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "source_pgid": "0.0", 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "ready_epoch": 0, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_started": 0, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "last_epoch_clean": 0, 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "source_version": "0'0", 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: "target_version": "0'0" 2024-05-08T17:19:49.622 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "last_change": "17", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend": "0", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_prenautilus": "0", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "last_force_op_resend_preluminous": "0", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "auid": 0, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_mode": "selfmanaged", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_seq": 0, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "snap_epoch": 0, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "pool_snaps": [], 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps": "[]", 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_bytes": 0, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "quota_max_objects": 0, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "tiers": [], 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "tier_of": -1, 2024-05-08T17:19:49.623 INFO:teuthology.orchestra.run.smithi154.stdout: "read_tier": -1, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "write_tier": -1, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_mode": "none", 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_bytes": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "target_max_objects": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_ratio_micro": 400000, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_dirty_high_ratio_micro": 600000, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_target_full_ratio_micro": 800000, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_flush_age": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "cache_min_evict_age": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profile": "", 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_params": { 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "none" 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_period": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_count": 0, 2024-05-08T17:19:49.624 INFO:teuthology.orchestra.run.smithi154.stdout: "use_gmt_hitset": true, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "min_read_recency_for_promote": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "min_write_recency_for_promote": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_grade_decay_rate": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "hit_set_search_last_n": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "grade_table": [], 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "stripe_width": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "expected_num_objects": 0, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "fast_read": false, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "options": {}, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "application_metadata": { 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "rbd": {} 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "read_balance": { 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "score_type": "Fair distribution", 2024-05-08T17:19:49.625 INFO:teuthology.orchestra.run.smithi154.stdout: "score_acting": 2.630000114440918, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "score_stable": 2.630000114440918, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "optimal_score": 1, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_acting": 2.630000114440918, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "raw_score_stable": 2.630000114440918, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity_weighted": 0.875, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity": 1, 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: "average_primary_affinity_weighted": 0.87999999523162842 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.626 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858", 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.627 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6808", 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6809", 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.629 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6811", 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6810", 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3590430155 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6808/3590430155", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6809/3590430155", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6811/3590430155", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6810/3590430155", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "7e66ef3b-5581-4f8c-9807-e16ea4ec972e", 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.631 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 39, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6812", 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.632 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6813", 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6815", 2024-05-08T17:19:49.633 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6814", 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2114037860 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6812/2114037860", 2024-05-08T17:19:49.634 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6813/2114037860", 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6815/2114037860", 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6814/2114037860", 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "86378b92-f863-41ec-a3a5-f1dc78645250", 2024-05-08T17:19:49.635 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 44, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.636 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6800", 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6801", 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6803", 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:49.639 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.640 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6802", 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2790378686 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6800/2790378686", 2024-05-08T17:19:49.641 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6801/2790378686", 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6803/2790378686", 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6802/2790378686", 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.642 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "2fa93019-16da-44c6-ac77-0ef554e24e10", 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.643 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 14, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 0, 2024-05-08T17:19:49.644 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.645 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6804", 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.646 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6805", 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6807", 2024-05-08T17:19:49.648 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.154:6806", 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3602122863 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.154:6804/3602122863", 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.154:6805/3602122863", 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.154:6807/3602122863", 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.154:6806/3602122863", 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.651 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "08750d7a-1ca6-4d68-a62e-893e8d979921", 2024-05-08T17:19:49.652 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:49.653 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 55, 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.654 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6812", 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.655 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6813", 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:49.656 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.657 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6815", 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.658 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6814", 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 1597208930 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.659 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6812/1597208930", 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6813/1597208930", 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6815/1597208930", 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6814/1597208930", 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.660 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "770b03a7-7f15-4427-81ce-e512b68bbb5c", 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 0, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 0, 2024-05-08T17:19:49.661 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 24, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.662 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6801", 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6803", 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6807", 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.665 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6805", 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3448847701 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6801/3448847701", 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6803/3448847701", 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6807/3448847701", 2024-05-08T17:19:49.666 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6805/3448847701", 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "exists" 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "ac82b5c6-7db8-4383-b0ac-0a203a4d84cd", 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:49.667 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 15, 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6800", 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6802", 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6806", 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.669 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6804", 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 3679914992 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6800/3679914992", 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6802/3679914992", 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6806/3679914992", 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6804/3679914992", 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.671 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: "uuid": "813f2f3d-68eb-487b-89a1-8b9a5a103485", 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: "up": 1, 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: "in": 1, 2024-05-08T17:19:49.672 INFO:teuthology.orchestra.run.smithi154.stdout: "weight": 1, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_affinity": 1, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_begin": 0, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "last_clean_end": 0, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "up_from": 13, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "up_thru": 42, 2024-05-08T17:19:49.673 INFO:teuthology.orchestra.run.smithi154.stdout: "down_at": 0, 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "lost_at": 0, 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addrs": { 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6808", 2024-05-08T17:19:49.674 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addrs": { 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.675 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6809", 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addrs": { 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.676 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6811", 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addrs": { 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: "addrvec": [ 2024-05-08T17:19:49.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: "type": "v2", 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: "addr": "172.21.15.183:6810", 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: "nonce": 2695818974 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "public_addr": "172.21.15.183:6808/2695818974", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "cluster_addr": "172.21.15.183:6809/2695818974", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_back_addr": "172.21.15.183:6811/2695818974", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "heartbeat_front_addr": "172.21.15.183:6810/2695818974", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "state": [ 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "exists", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "up" 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "osd_xinfo": [ 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 0, 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.679 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.817172+0000", 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 1, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.851798+0000", 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 2, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.765787+0000", 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 3, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.681 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.790359+0000", 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 4, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.824178+0000", 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.682 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 5, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.696786+0000", 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 24 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 6, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.683 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.761200+0000", 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "osd": 7, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "down_stamp": "0.000000", 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_probability": 0, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "laggy_interval": 0, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "features": 4540701547738038271, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "old_weight": 0, 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "last_purged_snaps_scrub": "2024-05-08T17:16:29.764552+0000", 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "dead_epoch": 0 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.684 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap": [ 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: "pgid": "2.3", 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: 4, 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: 5 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: "pgid": "2.7", 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: 2, 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: 3 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.685 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_items": [], 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_upmap_primaries": [], 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "pg_temp": [ 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "pgid": "2.3", 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "osds": [ 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: 4, 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: 2 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: ], 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "primary_temp": [], 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "blocklist": {}, 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "range_blocklist": {}, 2024-05-08T17:19:49.686 INFO:teuthology.orchestra.run.smithi154.stdout: "erasure_code_profiles": { 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "default": { 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "crush-failure-domain": "osd", 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "k": "2", 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "m": "1", 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "plugin": "jerasure", 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "technique": "reed_sol_van" 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "removed_snaps_queue": [], 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "new_removed_snaps": [], 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "new_purged_snaps": [], 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "crush_node_flags": {}, 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "device_class_flags": {}, 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode": { 2024-05-08T17:19:49.687 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_enabled": false, 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_bucket_count": 0, 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "degraded_stretch_mode": 0, 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "recovering_stretch_mode": 0, 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: "stretch_mode_bucket": 0 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:49.688 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:49.690 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:49.690 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:49.690 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:49.690 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:49.690 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:49.691 INFO:tasks.thrashosds.thrasher:j is {'epoch': 62, 'fsid': 'c69ba297-505d-4f4e-b390-bd22ef533cfc', 'created': '2024-05-08T17:16:22.335952+0000', 'modified': '2024-05-08T17:19:47.090999+0000', 'last_up_change': '2024-05-08T17:17:49.336612+0000', 'last_in_change': '2024-05-08T17:18:41.921552+0000', 'flags': 'sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit', 'flags_num': 5799936, 'flags_set': ['pglog_hardlimit', 'purged_snapdirs', 'recovery_deletes', 'sortbitwise'], 'crush_version': 4, 'full_ratio': 0.949999988079071, 'backfillfull_ratio': 0.8999999761581421, 'nearfull_ratio': 0.8500000238418579, 'cluster_snapshot': '', 'pool_max': 2, 'max_osd': 8, 'require_min_compat_client': 'luminous', 'min_compat_client': 'luminous', 'require_osd_release': 'squid', 'allow_crimson': False, 'pools': [{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T17:16:32.734319+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 6.900000095367432, 'score_stable': 6.900000095367432, 'optimal_score': 0.28999999165534973, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2024-05-08T17:16:33.268021+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.630000114440918, 'score_stable': 2.630000114440918, 'optimal_score': 1, 'raw_score_acting': 2.630000114440918, 'raw_score_stable': 2.630000114440918, 'primary_affinity_weighted': 0.875, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 0.8799999952316284}}], 'osds': [{'osd': 0, 'uuid': 'c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6808', 'nonce': 3590430155}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6809', 'nonce': 3590430155}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6811', 'nonce': 3590430155}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6810', 'nonce': 3590430155}]}, 'public_addr': '172.21.15.154:6808/3590430155', 'cluster_addr': '172.21.15.154:6809/3590430155', 'heartbeat_back_addr': '172.21.15.154:6811/3590430155', 'heartbeat_front_addr': '172.21.15.154:6810/3590430155', 'state': ['exists', 'up']}, {'osd': 1, 'uuid': '7e66ef3b-5581-4f8c-9807-e16ea4ec972e', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 39, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6812', 'nonce': 2114037860}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6813', 'nonce': 2114037860}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6815', 'nonce': 2114037860}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6814', 'nonce': 2114037860}]}, 'public_addr': '172.21.15.154:6812/2114037860', 'cluster_addr': '172.21.15.154:6813/2114037860', 'heartbeat_back_addr': '172.21.15.154:6815/2114037860', 'heartbeat_front_addr': '172.21.15.154:6814/2114037860', 'state': ['exists', 'up']}, {'osd': 2, 'uuid': '86378b92-f863-41ec-a3a5-f1dc78645250', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 44, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6800', 'nonce': 2790378686}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6801', 'nonce': 2790378686}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6803', 'nonce': 2790378686}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6802', 'nonce': 2790378686}]}, 'public_addr': '172.21.15.154:6800/2790378686', 'cluster_addr': '172.21.15.154:6801/2790378686', 'heartbeat_back_addr': '172.21.15.154:6803/2790378686', 'heartbeat_front_addr': '172.21.15.154:6802/2790378686', 'state': ['exists', 'up']}, {'osd': 3, 'uuid': '2fa93019-16da-44c6-ac77-0ef554e24e10', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 14, 'up_thru': 0, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6804', 'nonce': 3602122863}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6805', 'nonce': 3602122863}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6807', 'nonce': 3602122863}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.154:6806', 'nonce': 3602122863}]}, 'public_addr': '172.21.15.154:6804/3602122863', 'cluster_addr': '172.21.15.154:6805/3602122863', 'heartbeat_back_addr': '172.21.15.154:6807/3602122863', 'heartbeat_front_addr': '172.21.15.154:6806/3602122863', 'state': ['exists', 'up']}, {'osd': 4, 'uuid': '08750d7a-1ca6-4d68-a62e-893e8d979921', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 55, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6812', 'nonce': 1597208930}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6813', 'nonce': 1597208930}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6815', 'nonce': 1597208930}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6814', 'nonce': 1597208930}]}, 'public_addr': '172.21.15.183:6812/1597208930', 'cluster_addr': '172.21.15.183:6813/1597208930', 'heartbeat_back_addr': '172.21.15.183:6815/1597208930', 'heartbeat_front_addr': '172.21.15.183:6814/1597208930', 'state': ['exists', 'up']}, {'osd': 5, 'uuid': '770b03a7-7f15-4427-81ce-e512b68bbb5c', 'up': 0, 'in': 1, 'weight': 1, 'primary_affinity': 0, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 24, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6801', 'nonce': 3448847701}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6803', 'nonce': 3448847701}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6807', 'nonce': 3448847701}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6805', 'nonce': 3448847701}]}, 'public_addr': '172.21.15.183:6801/3448847701', 'cluster_addr': '172.21.15.183:6803/3448847701', 'heartbeat_back_addr': '172.21.15.183:6807/3448847701', 'heartbeat_front_addr': '172.21.15.183:6805/3448847701', 'state': ['exists']}, {'osd': 6, 'uuid': 'ac82b5c6-7db8-4383-b0ac-0a203a4d84cd', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 15, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6800', 'nonce': 3679914992}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6802', 'nonce': 3679914992}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6806', 'nonce': 3679914992}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6804', 'nonce': 3679914992}]}, 'public_addr': '172.21.15.183:6800/3679914992', 'cluster_addr': '172.21.15.183:6802/3679914992', 'heartbeat_back_addr': '172.21.15.183:6806/3679914992', 'heartbeat_front_addr': '172.21.15.183:6804/3679914992', 'state': ['exists', 'up']}, {'osd': 7, 'uuid': '813f2f3d-68eb-487b-89a1-8b9a5a103485', 'up': 1, 'in': 1, 'weight': 1, 'primary_affinity': 1, 'last_clean_begin': 0, 'last_clean_end': 0, 'up_from': 13, 'up_thru': 42, 'down_at': 0, 'lost_at': 0, 'public_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6808', 'nonce': 2695818974}]}, 'cluster_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6809', 'nonce': 2695818974}]}, 'heartbeat_back_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6811', 'nonce': 2695818974}]}, 'heartbeat_front_addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.183:6810', 'nonce': 2695818974}]}, 'public_addr': '172.21.15.183:6808/2695818974', 'cluster_addr': '172.21.15.183:6809/2695818974', 'heartbeat_back_addr': '172.21.15.183:6811/2695818974', 'heartbeat_front_addr': '172.21.15.183:6810/2695818974', 'state': ['exists', 'up']}], 'osd_xinfo': [{'osd': 0, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.817172+0000', 'dead_epoch': 0}, {'osd': 1, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.851798+0000', 'dead_epoch': 0}, {'osd': 2, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.765787+0000', 'dead_epoch': 0}, {'osd': 3, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.790359+0000', 'dead_epoch': 0}, {'osd': 4, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.824178+0000', 'dead_epoch': 0}, {'osd': 5, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.696786+0000', 'dead_epoch': 24}, {'osd': 6, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.761200+0000', 'dead_epoch': 0}, {'osd': 7, 'down_stamp': '0.000000', 'laggy_probability': 0, 'laggy_interval': 0, 'features': 4540701547738038271, 'old_weight': 0, 'last_purged_snaps_scrub': '2024-05-08T17:16:29.764552+0000', 'dead_epoch': 0}], 'pg_upmap': [{'pgid': '2.3', 'osds': [4, 5]}, {'pgid': '2.7', 'osds': [2, 3]}], 'pg_upmap_items': [], 'pg_upmap_primaries': [], 'pg_temp': [{'pgid': '2.3', 'osds': [4, 2]}], 'primary_temp': [], 'blocklist': {}, 'range_blocklist': {}, 'erasure_code_profiles': {'default': {'crush-failure-domain': 'osd', 'k': '2', 'm': '1', 'plugin': 'jerasure', 'technique': 'reed_sol_van'}}, 'removed_snaps_queue': [], 'new_removed_snaps': [], 'new_purged_snaps': [], 'crush_node_flags': {}, 'device_class_flags': {}, 'stretch_mode': {'stretch_mode_enabled': False, 'stretch_bucket_count': 0, 'degraded_stretch_mode': 0, 'recovering_stretch_mode': 0, 'stretch_mode_bucket': 0}} 2024-05-08T17:19:49.691 INFO:tasks.thrashosds.thrasher:Clearing pg_upmap on 2.3 2024-05-08T17:19:49.691 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd rm-pg-upmap 2.3 2024-05-08T17:19:49.691 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:49.693+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:49.692 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:49.790 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:49.793+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:49.797 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:49.797 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:49.798 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:49.798 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:49.798 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:49.807 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:49.891 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:49.895+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:49.952 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:49.953 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:49.953 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:49.953 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:49.961 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:49.992 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:49.995+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:50.092 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:50.096+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:50.098 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:19:50.098 INFO:teuthology.orchestra.run.smithi154.stderr:clear 2.3 pg_upmap mapping 2024-05-08T17:19:50.107 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:50.107 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:50.107 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:50.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '22 of 30' 2024-05-08T17:19:50.116 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:50.198 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:50.202+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:50.221 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:50.221 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:50.221 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:50.221 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:50.221 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.029685712000003, 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:50.222 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:50.223 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:50.224 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.225 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 76.026650193999998, 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.226 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.227 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.228 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.229 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:50.230 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:50.230 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.230 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.230 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.230 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:50.231 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.232 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:50.233 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:50.295 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:50.297+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:50.337 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:50.541 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:50.541 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:50.542 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:50.543 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:50.544 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:50.547+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:50.643 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:50.647+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:50.690 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:50.690 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:50.691 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:50.691 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:50.691 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:50.699 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:50.705 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:50.783 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:50.787+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:50.828 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:50.829 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:50.845 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:50.849+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:50.846 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:50.846 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:50.847 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:50.847 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 197.046896772, 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.848 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:50.849 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.850 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:50.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.854 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 197.013933181, 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:50.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:50.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.857 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.99616876600001, 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.860 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.865 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.99607573599999, 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:50.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:50.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.868 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.871 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.967977711, 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.872 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:50.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.96786763399999, 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:50.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:50.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.94400710599999, 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.883 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.888 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.802647498, 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.889 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:50.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:50.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:50.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.802530303, 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:50.894 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:50.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:50.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:50.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.802505352, 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.900 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:50.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.902 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:50.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:50.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40463449399999, 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:50.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:50.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:50.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.404618914, 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.911 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.916 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40459959399999, 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.917 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:50.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:50.922 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40452309099999, 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:50.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.926 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40450149599999, 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40447510199999, 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.933 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:50.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:50.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:50.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:50.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 196.40440087600001, 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:50.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:50.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:50.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 195.91860737900001, 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.944 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:50.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.948 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 195.91859409200001, 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.954 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 195.91852502399999, 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:50.955 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:50.960 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:50.961 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:50.962 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:50.962 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:50.967 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:50.967 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:50.968 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:50.968 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:50.968 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:50.968 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:50.969 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:50.969 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:50.969 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:50.970 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:50.970 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:50.970 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:50.965+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:51.063 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:51.066+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:51.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '23 of 30' 2024-05-08T17:19:51.118 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.118 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.127 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:51.164 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:51.168+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:51.264 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:51.268+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:51.280 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.280 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.290 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:51.365 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:51.369+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:51.437 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.437 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.446 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:51.466 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:51.470+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:51.552 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.552 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.561 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:51.567 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:51.571+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:51.665 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.665 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.667 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:51.671+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:51.674 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:51.768 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:51.772+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:51.821 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.821 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.830 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:51.871 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:51.874+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:51.971 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:51.974+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:51.975 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:51.975 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:51.984 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:52.072 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:52.076+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:52.090 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.090 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.100 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:52.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '24 of 30' 2024-05-08T17:19:52.116 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:19:52.173 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:52.176+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:52.209 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.209 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:52.302 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:52.306+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:52.369 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.369 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.374 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:52.378+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:52.378 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:52.476 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:52.478+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:52.527 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.527 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:52.579 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:52.583+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:52.642 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.642 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.651 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:52.696 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:52.699+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:52.768 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.768 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.778 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:52.778 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:52.780+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:52.879 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:52.882+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:52.924 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:52.924 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:52.933 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:52.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:52.979 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:52.982+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:53.038 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:53.038 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:53.047 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:53.079 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:53.083+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:53.096 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:53.096 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:53.111 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '25 of 30' 2024-05-08T17:19:53.112 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:19:53.115 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:53.115 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:53.133 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:53.133 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:53.151 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:53.151 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:53.170 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:53.170 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:53.170 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:53.180 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:53.184+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:53.194 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:53.194 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:53.203 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:53.281 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:53.284+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:53.308 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:53.308 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:53.317 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:53.382 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:53.385+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:53.408 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:53.408 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:53.425 INFO:teuthology.orchestra.run.smithi154.stdout:osd.7: {} 2024-05-08T17:19:53.426 INFO:teuthology.orchestra.run.smithi154.stderr:osd.7: osd_enable_op_tracker = '' 2024-05-08T17:19:53.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:53.430 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.273291076999996, 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:53.465 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.466 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.467 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.468 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 79.270255559000006, 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.469 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:53.470 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:53.471 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:53.472 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.473 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.473 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.473 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:53.474 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:19:53.517 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:53.520+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:53.578 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:53.578 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:53.578 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:53.578 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:53.584 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:53.587+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:53.587 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_blocked_ops 2024-05-08T17:19:53.685 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:53.688+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:53.734 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:53.734 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:53.734 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:53.743 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_historic_ops 2024-05-08T17:19:53.786 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:53.790+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:53.850 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:53.850 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:1 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [create,setxattr striper.version (16) in=15b,setxattr striper.excl in=12b,setxattr striper.size (16) in=12b,setxattr striper.allocated (16) in=17b,setxattr striper.layout.stripe_unit (16) in=26b,setxattr striper.layout.stripe_count (16) in=27b,setxattr striper.layout.object_size (16) in=26b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 200.049805647, 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010111561, 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.851 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 1 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802350+0000", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.852 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802352+0000", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7439999999999999e-06 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802372+0000", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9814e-05 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802382+0000", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0370999999999999e-05 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.853 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802394+0000", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1960000000000001e-05 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802443+0000", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.8508999999999999e-05 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for peered", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.802450+0000", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1980000000000004e-06 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.854 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804701+0000", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022512769999999999 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.804905+0000", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00020393399999999999 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.805098+0000", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00019307499999999999 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.855 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.806309+0000", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012105799999999999 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812390+0000", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0060813819999999998 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812433+0000", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2786e-05 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.856 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.812462+0000", 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.8931000000000001e-05 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:22 1.0 1:30306672:devicehealth::main.db.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 200.016842056, 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0091440489999999996, 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.857 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 22 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835314+0000", 2024-05-08T17:19:53.858 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835315+0000", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.4499999999999996e-07 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835319+0000", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4549999999999997e-06 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835326+0000", 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0559999999999999e-06 2024-05-08T17:19:53.859 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.835332+0000", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6629999999999998e-06 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836372+0000", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010402160000000001 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836424+0000", 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1242999999999997e-05 2024-05-08T17:19:53.860 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.836504+0000", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.0500999999999994e-05 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.837620+0000", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001116218 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844306+0000", 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0066856160000000001 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.861 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844393+0000", 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7514000000000003e-05 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.844458+0000", 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4721999999999995e-05 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:35 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.862 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.99907764100001, 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.006397079, 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 35 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.863 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853078+0000", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853079+0000", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3099999999999999e-07 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853141+0000", 2024-05-08T17:19:53.864 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2236000000000005e-05 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853146+0000", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0860000000000001e-06 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853150+0000", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9260000000000002e-06 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855215+0000", 2024-05-08T17:19:53.865 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0020648609999999999 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855266+0000", 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1116000000000003e-05 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.855353+0000", 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7239999999999998e-05 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.866 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.858356+0000", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030031469999999998 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859418+0000", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0010612340000000001 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859442+0000", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4613e-05 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.867 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859476+0000", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.3389000000000002e-05 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:37 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.998984611, 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083231759999999998, 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.868 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 37 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853171+0000", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853172+0000", 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.6199999999999999e-07 2024-05-08T17:19:53.869 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853174+0000", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1789999999999998e-06 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853177+0000", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9890000000000001e-06 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.853180+0000", 2024-05-08T17:19:53.870 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5779999999999999e-06 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856075+0000", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002894504 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856127+0000", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2339999999999997e-05 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.856235+0000", 2024-05-08T17:19:53.871 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010767799999999999 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.859542+0000", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0033074670000000001 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861424+0000", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0018811240000000001 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861455+0000", 2024-05-08T17:19:53.872 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1507999999999997e-05 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.861495+0000", 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9546999999999999e-05 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:55 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 516~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.97088658600001, 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096343980000000006, 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.873 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 55 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.874 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881270+0000", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3200000000000005e-07 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881346+0000", 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.6412e-05 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.875 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881359+0000", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.2536e-05 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881367+0000", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8630000000000008e-06 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882052+0000", 2024-05-08T17:19:53.876 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00068521900000000004 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882101+0000", 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9026999999999997e-05 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882205+0000", 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00010429699999999999 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.877 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883057+0000", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085137099999999996 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890788+0000", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077308589999999996 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890879+0000", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.1044000000000004e-05 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.878 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890904+0000", 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5238000000000001e-05 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:56 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.97077650899999, 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0095678699999999992, 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.879 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 56 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.880 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881380+0000", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.4199999999999995e-07 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881385+0000", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3479999999999997e-06 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881390+0000", 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.6450000000000002e-06 2024-05-08T17:19:53.881 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.881397+0000", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3799999999999999e-06 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882540+0000", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.001143451 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882586+0000", 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6279999999999997e-05 2024-05-08T17:19:53.882 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.882659+0000", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2442999999999997e-05 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.883293+0000", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00063414899999999998 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890909+0000", 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076160439999999998 2024-05-08T17:19:53.883 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890931+0000", 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2243e-05 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.890947+0000", 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6245000000000001e-05 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.884 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:70 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66052~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.94691598099999, 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087538019999999998, 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 70 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.885 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905240+0000", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905241+0000", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.2300000000000001e-07 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.886 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905245+0000", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905269+0000", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4053000000000002e-05 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.905278+0000", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.8999999999999995e-06 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.887 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908362+0000", 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0030839579999999999 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908416+0000", 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.4277000000000002e-05 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.908489+0000", 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2909000000000003e-05 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.888 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.909190+0000", 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00070129600000000002 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913922+0000", 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0047318480000000003 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.889 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913954+0000", 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.2518999999999998e-05 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:33.913994+0000", 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9539999999999998e-05 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:115 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.890 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.805556373, 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0065187819999999999, 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 115 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:53.891 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046600+0000", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.75e-07 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046688+0000", 2024-05-08T17:19:53.892 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7427000000000002e-05 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046696+0000", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.3450000000000006e-06 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046703+0000", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.2200000000000003e-06 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051217+0000", 2024-05-08T17:19:53.893 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045135030000000003 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051294+0000", 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.7138000000000006e-05 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.051413+0000", 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000119632 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.894 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052889+0000", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0014758079999999999 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053075+0000", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00018592899999999999 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053098+0000", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3149000000000001e-05 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.895 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053119+0000", 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.0156000000000001e-05 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:116 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.805439178, 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00725736, 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.896 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 116 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046717+0000", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.897 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046718+0000", 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.8800000000000002e-07 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046722+0000", 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1840000000000001e-06 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046728+0000", 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6340000000000003e-06 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.898 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046733+0000", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9239999999999998e-06 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052264+0000", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0055305160000000001 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052309+0000", 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5327000000000002e-05 2024-05-08T17:19:53.899 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052378+0000", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8959000000000002e-05 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053143+0000", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00076525300000000003 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053918+0000", 2024-05-08T17:19:53.900 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00077511699999999997 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053949+0000", 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1047000000000003e-05 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053974+0000", 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4811e-05 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.901 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:117 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [setxattr striper.size (16) in=12b,setxattr striper.version (16) in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.805414227, 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0072785970000000004, 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.902 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 117 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.903 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046742+0000", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3799999999999998e-07 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046746+0000", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9820000000000002e-06 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046752+0000", 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3170000000000001e-06 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.904 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.046757+0000", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.95e-06 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052581+0000", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0058241719999999999 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052627+0000", 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5785999999999998e-05 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.905 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.052700+0000", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3807000000000007e-05 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053472+0000", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000771345 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.053979+0000", 2024-05-08T17:19:53.906 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00050766400000000001 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054003+0000", 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3499999999999999e-05 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.054020+0000", 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.7635999999999999e-05 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.907 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:135 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.407543369, 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0083666890000000001, 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 135 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.908 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444613+0000", 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.7599999999999998e-07 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.909 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444617+0000", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8870000000000002e-06 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444621+0000", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8789999999999996e-06 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444624+0000", 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.709e-06 2024-05-08T17:19:53.910 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.446965+0000", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0023409329999999999 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447006+0000", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.1019999999999997e-05 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447093+0000", 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.6997999999999996e-05 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.911 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452882+0000", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0057882899999999998 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452925+0000", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3711000000000002e-05 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452956+0000", 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0392e-05 2024-05-08T17:19:53.912 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.452979+0000", 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3594000000000001e-05 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:136 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131600~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.407527789, 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084735030000000003, 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.913 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 136 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.914 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444628+0000", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444629+0000", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4400000000000001e-07 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444635+0000", 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.1249999999999998e-06 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.915 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444638+0000", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9270000000000001e-06 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444640+0000", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.5490000000000001e-06 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447294+0000", 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0026535999999999999 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.916 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447341+0000", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.723e-05 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447412+0000", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0568999999999995e-05 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453020+0000", 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0056086340000000004 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.917 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453027+0000", 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8469999999999998e-06 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453065+0000", 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7902999999999999e-05 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.918 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453102+0000", 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.6875000000000002e-05 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:137 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131604~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.40750846899999, 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0098701290000000001, 2024-05-08T17:19:53.919 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 137 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.920 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444648+0000", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.53e-07 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444700+0000", 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2465e-05 2024-05-08T17:19:53.921 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444710+0000", 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.2760000000000006e-06 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444715+0000", 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9139999999999999e-06 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.922 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447851+0000", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0031362489999999998 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447896+0000", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5154000000000002e-05 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.447966+0000", 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0344000000000003e-05 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.923 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453135+0000", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0051686900000000001 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454452+0000", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0013167669999999999 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454483+0000", 2024-05-08T17:19:53.924 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.1424999999999998e-05 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454518+0000", 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.4591999999999997e-05 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:138 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197140~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:53.925 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.40743196599999, 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010234617, 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 138 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.926 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444724+0000", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444725+0000", 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0699999999999997e-07 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.927 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444728+0000", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.732e-06 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444733+0000", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.0080000000000002e-06 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444738+0000", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9089999999999999e-06 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.928 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453160+0000", 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0084215379999999992 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453211+0000", 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1007000000000001e-05 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453280+0000", 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.9387000000000005e-05 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.929 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454903+0000", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0016229969999999999 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454910+0000", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.6510000000000004e-06 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454934+0000", 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4199999999999999e-05 2024-05-08T17:19:53.930 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454959+0000", 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4681e-05 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:139 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197144~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.407410371, 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.010373874999999999, 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.931 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 139 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:53.932 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444746+0000", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.84e-07 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444750+0000", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5310000000000002e-06 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444757+0000", 2024-05-08T17:19:53.933 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8669999999999996e-06 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444761+0000", 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5979999999999999e-06 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453489+0000", 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0087276539999999996 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.934 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453567+0000", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.8560999999999995e-05 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453694+0000", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00012643499999999999 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454964+0000", 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0012706 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.935 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455018+0000", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.3378000000000003e-05 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455079+0000", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0884999999999997e-05 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455120+0000", 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.0982000000000002e-05 2024-05-08T17:19:53.936 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:140 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 197148~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.40738397699999, 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013067663, 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 140 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.937 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444772+0000", 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.7399999999999999e-07 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.938 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444825+0000", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.2403e-05 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444831+0000", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3049999999999998e-06 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444837+0000", 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.5799999999999999e-06 2024-05-08T17:19:53.939 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453932+0000", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0090947419999999994 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.453971+0000", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.9285000000000001e-05 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454034+0000", 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.3569000000000001e-05 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.940 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455149+0000", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0011148499999999999 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457785+0000", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002635971 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457816+0000", 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.0765000000000001e-05 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.941 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457840+0000", 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.3819e-05 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:141 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 262684~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e15)", 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 199.40730975100001, 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.013184278000000001, 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.942 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 141 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444846+0000", 2024-05-08T17:19:53.943 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444847+0000", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.82e-07 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444850+0000", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.5130000000000001e-06 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444856+0000", 2024-05-08T17:19:53.944 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.0630000000000002e-06 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.444861+0000", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5090000000000004e-06 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454550+0000", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0096893229999999997 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454595+0000", 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4295000000000002e-05 2024-05-08T17:19:53.945 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.454665+0000", 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0140000000000003e-05 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.455409+0000", 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00074431300000000003 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.946 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.457989+0000", 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.002579755 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458012+0000", 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.319e-05 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.458031+0000", 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8695e-05 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.947 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:157 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66056~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 198.92151625400001, 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0076618739999999999, 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 157 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.948 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930640+0000", 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1799999999999999e-07 2024-05-08T17:19:53.949 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930644+0000", 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.8179999999999997e-06 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930647+0000", 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.841e-06 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.950 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930650+0000", 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9189999999999999e-06 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932636+0000", 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0019865690000000001 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932679+0000", 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.2622e-05 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.951 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932751+0000", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.1874999999999999e-05 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933670+0000", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00091974899999999998 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938248+0000", 2024-05-08T17:19:53.952 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0045776499999999999 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938277+0000", 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9396000000000001e-05 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938302+0000", 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4216999999999999e-05 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.953 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:158 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 66060~65536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 198.92150296700001, 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.007736026, 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 158 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.954 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930653+0000", 2024-05-08T17:19:53.955 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.1299999999999999e-07 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930703+0000", 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.9988000000000003e-05 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930709+0000", 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.1769999999999997e-06 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.956 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930713+0000", 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.4159999999999997e-06 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.932965+0000", 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0022525520000000001 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933039+0000", 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.3683000000000006e-05 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.957 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933160+0000", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000120426 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934016+0000", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00085596999999999995 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938338+0000", 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0043224029999999998 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.958 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938367+0000", 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.9094e-05 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938389+0000", 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.2104e-05 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "osd_op(client.4215.0:159 1.0 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head [write 131596~4] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e16)", 2024-05-08T17:19:53.959 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 198.92143389899999, 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0077314410000000004, 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_info": { 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client": "client.4215", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "client_addr": "172.21.15.154:0/397872759", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "tid": 159 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:53.960 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930722+0000", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930723+0000", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.3300000000000003e-07 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930726+0000", 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 3.568e-06 2024-05-08T17:19:53.961 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930732+0000", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 5.614e-06 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.930736+0000", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.5410000000000002e-06 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933708+0000", 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0029712530000000001 2024-05-08T17:19:53.962 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933754+0000", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 4.6193000000000003e-05 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "waiting for subops from 0", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.933825+0000", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.0909999999999997e-05 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "op_commit", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.934310+0000", 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00048523099999999998 2024-05-08T17:19:53.963 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "sub_op_commit_rec", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938412+0000", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.0041017190000000002 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "commit_sent", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938436+0000", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 2.4239e-05 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:16:34.938454+0000", 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.774e-05 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.964 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:53.965 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:53.965 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:53.966 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:53.969+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:53.967 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:54.067 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:54.071+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:54.075 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.075 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:54.075 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:54.075 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.084 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:54.113 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '26 of 30' 2024-05-08T17:19:54.197 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:54.200+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:54.260 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.260 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:54.260 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:54.260 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:54.261 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.270 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:54.273+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:54.271 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:54.370 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:54.374+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.421 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.61678697799999, 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.422 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.423 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.424 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:54.425 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:54.426 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.427 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:54.428 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.59481382600001, 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:54.429 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.430 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.431 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.432 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.433 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.434 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.59435747000001, 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.435 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.436 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.437 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.438 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.439 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:54.440 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:54.441 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.575315714, 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.442 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.443 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.444 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.445 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.56702665500001, 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.446 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:54.447 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:54.448 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.449 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.566538651, 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.450 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:54.451 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:54.452 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:54.453 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:54.454 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.56606421800001, 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.455 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:54.456 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:54.457 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.458 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.56567084299999, 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:54.460 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:54.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:54.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.559484406, 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.55468095500001, 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.467 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:54.470 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.55210549700001, 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.475 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.549319801, 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:54.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.479 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.544317178, 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:54.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:54.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 200.37314111699999, 2024-05-08T17:19:54.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:54.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.96255714700001, 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:54.490 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:54.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.48976003600001, 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:54.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:54.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.48917446999999, 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:54.497 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.501 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.48873506300001, 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:54.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:54.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.48808707800001, 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 199.482625351, 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.510 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:54.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:54.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:54.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.514 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.516 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:54.516 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:54.519+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:54.617 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:54.620+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:54.662 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.662 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:54.662 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:54.662 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.671 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:54.755 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:54.759+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:54.818 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:19:54.821+0000 7f6df686e640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 7 (PID: 32510) UID: 0 2024-05-08T17:19:54.822 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.823 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:54.823 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:54.823 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:54.823 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.832 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:54.919 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:54.923+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:54.979 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 126.700557792, 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.980 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.981 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.982 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.983 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:19:54.984 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 64.638553897999998, 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:54.985 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.986 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.987 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.988 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 4.6371887950000001, 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000241578, 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 6 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.989 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:54.990 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345668+0000", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.858e-06 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345685+0000", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7263000000000001e-05 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345698+0000", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3237e-05 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345707+0000", 2024-05-08T17:19:54.991 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3639999999999999e-06 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345789+0000", 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2301999999999999e-05 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345908+0000", 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118554 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:54.992 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:54.993 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:54.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:55.035 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:55.039+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:55.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '27 of 30' 2024-05-08T17:19:55.115 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5] live_osds: [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:55.116 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:19:55.116 INFO:tasks.thrashosds.thrasher:Killing osd 7, live_osds are [0, 1, 2, 3, 4, 6, 7] 2024-05-08T17:19:55.116 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T17:19:55.116 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:19:55.128 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:19:55.145 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:55.145 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:55.145 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:55.145 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:55.146 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:55.148+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:55.155 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:55.159 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T17:19:55.159 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.7 2024-05-08T17:19:55.159 INFO:tasks.thrashosds.thrasher:remote for osd 7 is ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:19:55.159 INFO:tasks.thrashosds.thrasher:remote for exp osd 5 is ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:19:55.159 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-05-08T17:19:55.176 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T17:19:55.176 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op list-pgs 2024-05-08T17:19:55.239 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:55.243+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:55.318 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:55.319 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:55.319 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:55.319 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:55.319 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:55.322 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:55.325+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:55.328 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:55.423 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:55.427+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:55.431 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:19:55.477 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:55.478 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:55.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:55.478 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:55.478 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.479 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:19:55.479 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:55.479 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 127.19862476599999, 2024-05-08T17:19:55.479 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:19:55.479 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:55.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:55.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:55.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:55.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:19:55.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:19:55.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:55.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:55.487 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:55.488 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:55.488 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:55.489 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:55.546 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:55.548+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:55.557 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:55.557 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:55.575 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:55.575 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:55.592 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:55.592 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:55.609 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:55.610 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:55.610 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:55.610 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:55.620 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:55.625 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:55.628+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:55.627 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:55.628 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:55.628 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:55.646 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:55.646 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:55.726 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:55.729+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:55.733 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:55.733 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:55.742 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:55.827 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:55.830+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:55.888 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:55.888 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:55.897 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:55.928 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:55.930+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:56.007 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.007 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.016 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:56.028 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:56.031+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:56.106 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:19:56.109 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:19:56.113 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:56.114 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:56.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '28 of 30' 2024-05-08T17:19:56.123 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.123 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.132 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:56.164 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:56.166+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:56.230 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:56.233+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:56.235 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.236 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.245 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:56.327 INFO:teuthology.orchestra.run.smithi183.stdout:2.3 2024-05-08T17:19:56.327 INFO:teuthology.orchestra.run.smithi183.stdout:2.2 2024-05-08T17:19:56.331 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:56.333+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:56.352 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.352 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.361 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:56.432 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:56.434+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:56.466 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.466 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.475 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:56.534 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:56.536+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:56.579 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.579 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.587 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:56.634 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:56.637+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:56.694 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.694 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.703 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:56.744 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:56.747+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:56.836 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:56.839+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:56.848 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.848 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.852 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --op export --pgid 2.3 --file /var/log/ceph/exp.2.3.5 2024-05-08T17:19:56.857 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:56.937 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:56.939+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:56.963 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:56.963 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:56.971 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:57.037 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:57.040+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:57.078 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.078 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:57.118 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log '29 of 30' 2024-05-08T17:19:57.161 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:57.163+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:57.239 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:57.242+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:57.268 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.268 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.278 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:57.362 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:57.365+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:57.425 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.425 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.434 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:19:57.476 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:57.478+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:57.542 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:57.545+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:57.581 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.581 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.590 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:19:57.643 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:57.646+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:57.736 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.736 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.745 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:19:57.745 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:57.747+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:57.846 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:57.848+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:57.850 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.851 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.860 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:19:57.946 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:57.949+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:57.966 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:57.967 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:57.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:19:57.979 INFO:teuthology.orchestra.run.smithi183.stderr:Exporting 2.3 info 2.3( empty local-lis/les=19/20 n=0 ec=15/15 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-05-08T17:19:57.980 INFO:teuthology.orchestra.run.smithi183.stderr:Export successful 2024-05-08T17:19:58.047 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:58.049+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:58.082 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:58.083 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:58.090 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:19:58.115 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:19:58.119 INFO:tasks.mon_recovery:adding mon c back in 2024-05-08T17:19:58.119 INFO:tasks.ceph.mon.c:Restarting daemon 2024-05-08T17:19:58.119 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2024-05-08T17:19:58.122 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:19:58.148 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:58.151+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:58.204 INFO:tasks.ceph.mon.c:Started 2024-05-08T17:19:58.204 INFO:tasks.mon_recovery.ceph_manager:waiting for quorum size 3 2024-05-08T17:19:58.205 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:19:58.276 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:58.276 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:58.276 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:58.279+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:58.287 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:19:58.308 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:19:58.308 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:19:58.332 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:19:58.332 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:19:58.349 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:58.352+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:58.358 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:19:58.358 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:19:58.389 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:19:58.390 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:19:58.400 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:19:58.400 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:19:58.407 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:19:58.407 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:19:58.408 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:19:58.409 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:19:58.425 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:19:58.426 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:19:58.426 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:19:58.430 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:19:58.431 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:19:58.471 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:58.474+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:58.511 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-5 --journal-path /var/lib/ceph/osd/ceph-5/journal --force --op remove --pgid 2.3 2024-05-08T17:19:58.550 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:58.553+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:58.557 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:58.557 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:58.557 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:58.557 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:58.557 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:58.567 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:19:58.625 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":42,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":30,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:19:58.652 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:58.655+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:19:58.670 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:58.670 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:58.670 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:58.671 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:19:58.671 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:58.679 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:19:58.753 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:19:58.756+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:19:58.825 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:58.825 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:58.825 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:19:58.825 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:58.835 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:19:58.853 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:58.856+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:58.950 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:58.950 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:19:58.950 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:19:58.950 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:19:58.950 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:58.955 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:58.957+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:58.959 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:19:59.056 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:59.059+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:59.066 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:19:59.066 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.874989803000005, 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:59.067 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:59.068 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:59.069 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:59.070 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 84.871954285000001, 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:59.071 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:19:59.072 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:19:59.073 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:19:59.074 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:19:59.075 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:19:59.076 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:19:59.108 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:19:59.166 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:59.168+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:59.186 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:59.186 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:59.186 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:59.195 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:19:59.257 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:19:59.260+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:19:59.342 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:59.342 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:59.342 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:59.342 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:59.342 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:59.351 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:19:59.394 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:59.397+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:19:59.459 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.655308273, 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.460 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.461 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.462 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:19:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:19:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.463 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.464 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:19:59.465 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:19:59.466 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.467 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.468 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.63333512099999, 2024-05-08T17:19:59.469 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.470 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:59.471 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:19:59.472 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.473 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.474 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.475 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:19:59.476 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.477 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.478 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.478 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:19:59.478 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:19:59.478 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.478 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:59.479 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.63287876499999, 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.480 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.481 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:59.482 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.482 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.482 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.482 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.482 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:19:59.483 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:19:59.484 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.485 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.486 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.487 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:19:59.488 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.61383700900001, 2024-05-08T17:19:59.489 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:59.490 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.491 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:19:59.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:19:59.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.60554794999999, 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:19:59.495 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:59.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.499 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.60505994600001, 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:19:59.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:19:59.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.60458551299999, 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.506 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.507 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:19:59.508 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.604192138, 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:59.509 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.510 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:19:59.511 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.512 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.513 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.59800570100001, 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.514 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.515 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.516 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:19:59.517 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.518 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.519 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.59320224999999, 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.520 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.521 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:19:59.522 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:19:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.523 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.524 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.525 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.59062679199999, 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.526 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.527 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.528 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.529 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.530 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.58784109600001, 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:59.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:19:59.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:19:59.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:19:59.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.58283847300001, 2024-05-08T17:19:59.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:19:59.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.539 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.411662412, 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:59.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:19:59.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.543 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 205.00107844199999, 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.544 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.545 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.546 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.547 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 204.52828133099999, 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.548 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:19:59.549 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.550 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.551 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:19:59.552 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 204.527695765, 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:19:59.553 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:19:59.554 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:19:59.555 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.556 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 204.52725635799999, 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.557 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:19:59.558 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:19:59.559 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:19:59.560 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 204.52660837299999, 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.561 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.562 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.563 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.564 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 204.52114664600001, 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:19:59.565 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:19:59.566 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:19:59.567 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:19:59.568 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:19:59.569 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:59.571 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:19:59.571 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:59.573+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:59.606 INFO:teuthology.orchestra.run.smithi183.stdout: marking collection for removal 2024-05-08T17:19:59.606 INFO:teuthology.orchestra.run.smithi183.stdout:setting '_remove' omap key 2024-05-08T17:19:59.606 INFO:teuthology.orchestra.run.smithi183.stdout:finish_remove_pgs 2.3_head removing 2.3 2024-05-08T17:19:59.611 INFO:teuthology.orchestra.run.smithi183.stdout:Remove successful 2024-05-08T17:19:59.671 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:19:59.674+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:19:59.718 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:59.718 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:59.718 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:19:59.718 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:59.727 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:19:59.811 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:19:59.814+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:19:59.873 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:19:59.876+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:19:59.873 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:19:59.874 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:19:59.874 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:19:59.874 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:19:59.874 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:19:59.882 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:19:59.974 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:19:59.977+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:00.028 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 131.74955105999999, 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:00.029 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.030 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.031 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:20:00.032 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.033 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 69.687547166000002, 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:00.034 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.035 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.036 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:00.037 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 9.6861820630000004, 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000241578, 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 6 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:00.038 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345668+0000", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.858e-06 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345685+0000", 2024-05-08T17:20:00.039 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7263000000000001e-05 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345698+0000", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3237e-05 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345707+0000", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3639999999999999e-06 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345789+0000", 2024-05-08T17:20:00.040 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2301999999999999e-05 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345908+0000", 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118554 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.041 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:00.043 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:20:00.075 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:00.078+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.125 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-05-08T17:20:00.151 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:00.151 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:00.151 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:00.160 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:20:00.210 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:00.213+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.266 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:00.266 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:00.266 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:00.266 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:00.266 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:00.275 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:20:00.276 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:00.279+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.378 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:00.380+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:00.381 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:00.381 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:00.381 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:00.381 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 132.10310230600001, 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.382 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.383 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.384 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:00.385 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:00.391 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:20:00.431 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:20:00.479 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:00.482+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.541 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:00.541 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:00.541 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:00.550 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:20:00.557 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:20:00.557 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:20:00.575 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:20:00.575 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:20:00.580 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:00.583+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:00.592 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:20:00.592 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:20:00.609 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:20:00.609 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:20:00.627 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:20:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:20:00.627 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:20:00.645 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:20:00.645 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:20:00.645 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:20:00.649 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:20:00.650 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:20:00.659 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:00.659 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:00.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:20:00.710 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:00.712+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:00.782 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:00.785+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:00.816 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:00.816 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:00.825 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:20:00.884 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:00.887+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:00.931 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:00.931 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:00.940 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:20:00.984 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:00.987+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:01.085 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:01.088+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:01.085 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.086 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.094 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:20:01.124 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set noscrub 2024-05-08T17:20:01.187 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:01.189+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:01.239 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.239 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.248 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:20:01.280 INFO:teuthology.orchestra.run.smithi183.stdout:2.0 2024-05-08T17:20:01.281 INFO:teuthology.orchestra.run.smithi183.stdout:2.5 2024-05-08T17:20:01.281 INFO:teuthology.orchestra.run.smithi183.stdout:1.0 2024-05-08T17:20:01.287 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:01.289+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:01.356 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.357 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.365 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:20:01.388 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:01.390+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:01.489 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:01.492+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:01.510 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.511 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.519 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:20:01.603 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:01.606+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:01.640 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:20:01.664 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.665 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:20:01.715 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:01.718+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:01.792 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.792 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.793 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:01.794+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:01.800 INFO:tasks.thrashosds.thrasher:Moving pg 2.3 from osd.5 to osd.7 2024-05-08T17:20:01.800 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.3.5 2024-05-08T17:20:01.801 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:20:01.893 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:01.895+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:01.907 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:01.907 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:01.916 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:20:01.993 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:01.995+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:02.006 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":42,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":33,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:20:02.023 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.023 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:20:02.116 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:02.119+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:02.122 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is set 2024-05-08T17:20:02.181 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.181 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.190 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:20:02.195 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:02.198+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:02.296 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:02.298+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:02.296 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.296 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.305 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:20:02.397 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:02.400+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:02.452 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.452 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.461 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:20:02.499 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:02.501+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:02.600 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:02.603+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:02.607 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.607 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.616 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:20:02.650 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:20:02.703 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:02.706+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:02.725 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:02.725 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:02.735 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:20:02.784 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:20:02.785 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:20:02.802 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:02.804+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:02.804 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:20:02.804 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:20:02.821 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:20:02.821 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:20:02.838 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:20:02.838 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:20:02.855 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:20:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:20:02.856 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:20:02.873 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:20:02.873 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:20:02.874 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:20:02.877 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:20:02.877 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:20:02.884 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:02.884 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:02.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 134.60607941200001, 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:20:02.885 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.886 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.887 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:02.888 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:02.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:02.889 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:02.889 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:02.889 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:02.894 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:20:02.930 INFO:teuthology.orchestra.run.smithi183.stdout:get_pg_num_history pg_num_history pg_num_history(e66 pg_nums {1={14=1},2={15=8}} deleted_pools ) 2024-05-08T17:20:02.930 INFO:teuthology.orchestra.run.smithi183.stdout:Importing pgid 2.3 2024-05-08T17:20:02.930 INFO:teuthology.orchestra.run.smithi183.stdout:write_pg epoch 20 info 2.3( empty local-lis/les=19/20 n=0 ec=15/15 lis/c=19/19 les/c/f=20/20/0 sis=19) 2024-05-08T17:20:02.935 INFO:teuthology.orchestra.run.smithi183.stdout:Import successful 2024-05-08T17:20:02.936 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:02.938+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:03.005 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:03.007+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:03.041 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:03.041 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:03.041 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:03.042 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:03.050 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:20:03.105 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:03.107+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:03.159 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:03.159 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:03.159 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:03.160 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:03.160 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:03.168 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:20:03.206 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:03.209+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:03.307 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:03.310+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:03.315 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:03.315 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:03.315 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:03.316 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:20:03.316 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:03.324 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:20:03.408 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:03.411+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:03.432 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:03.432 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:03.432 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:03.432 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:03.441 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:20:03.453 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /var/log/ceph/exp.2.3.5 2024-05-08T17:20:03.508 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:03.512+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:03.549 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:03.549 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:03.549 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:20:03.549 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:20:03.549 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:03.558 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:20:03.610 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:03.612+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:03.703 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:03.703 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:20:03.703 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:20:03.703 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:20:03.703 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:03.711 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:03.713+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:03.712 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:20:03.811 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:03.814+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:03.817 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:03.817 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:03.817 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:03.817 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:03.826 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:20:03.912 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:03.915+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:03.970 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:20:03.971 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:20:03.971 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:03.979 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:20:04.020 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:04.023+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:04.114 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:04.117+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:04.124 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [ 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head, version: 16'116, data_included: [0~4096,65536~197152], data_size: 201248, omap_header_size: 0, omap_entries_size: 0, attrset_size: 9, recovery_info: ObjectRecoveryInfo(1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head@16'116, size: 262688, copy_subset: [(0, 262688)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 262688, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 89.932521065000003, 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00093846000000000001, 2024-05-08T17:20:04.125 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195508+0000", 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.126 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195509+0000", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 7.4799999999999997e-07 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195724+0000", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00021542999999999999 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195743+0000", 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.8496e-05 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.127 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195757+0000", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.4594e-05 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195826+0000", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 6.8272e-05 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.195837+0000", 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.1245e-05 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.128 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.196446+0000", 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060967499999999995 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: "description": "MOSDPGPush(1.0 34/33 [PushOp(1:30306672:devicehealth::main.db.0000000000000000:head, version: 16'117, data_included: [0~327680], data_size: 327680, omap_header_size: 0, omap_entries_size: 0, attrset_size: 10, recovery_info: ObjectRecoveryInfo(1:30306672:devicehealth::main.db.0000000000000000:head@16'117, size: 327680, copy_subset: [(0, 327680)], clone_subset: {}, snapset: 0=[]:{}, object_exist: false), after_progress: ObjectRecoveryProgress(!first, data_recovered_to: 327680, data_complete: true, omap_recovered_to: , omap_complete: true, error: false), before_progress: ObjectRecoveryProgress(first, data_recovered_to: 0, data_complete: false, omap_recovered_to: , omap_complete: false, error: false))])", 2024-05-08T17:20:04.129 INFO:teuthology.orchestra.run.smithi183.stdout: "initiated_at": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "age": 89.929485546999999, 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00097515899999999997, 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "continuous": false, 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "type_data": { 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "flag_point": "started", 2024-05-08T17:20:04.130 INFO:teuthology.orchestra.run.smithi183.stdout: "events": [ 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "initiated", 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.131 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "header_read", 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "throttled", 2024-05-08T17:20:04.132 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198544+0000", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 8.7599999999999996e-07 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "all_read", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198771+0000", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.000226985 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "dispatched", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198791+0000", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.9568e-05 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.133 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198807+0000", 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.6458000000000001e-05 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "reached_pg", 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198901+0000", 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 9.3571000000000002e-05 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "started", 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.198911+0000", 2024-05-08T17:20:04.134 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 1.0387999999999999e-05 2024-05-08T17:20:04.135 INFO:teuthology.orchestra.run.smithi183.stdout: }, 2024-05-08T17:20:04.135 INFO:teuthology.orchestra.run.smithi183.stdout: { 2024-05-08T17:20:04.135 INFO:teuthology.orchestra.run.smithi183.stdout: "event": "done", 2024-05-08T17:20:04.135 INFO:teuthology.orchestra.run.smithi183.stdout: "time": "2024-05-08T17:18:34.199519+0000", 2024-05-08T17:20:04.135 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 0.00060731299999999995 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: } 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout: ] 2024-05-08T17:20:04.136 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:04.136 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd set nodeep-scrub 2024-05-08T17:20:04.138 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:20:04.215 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:04.218+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:04.258 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:04.258 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:04.259 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:04.259 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:04.269 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:20:04.317 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:04.320+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:04.415 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:04.416 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:04.416 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:04.416 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:04.416 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:04.417 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:04.419+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:04.424 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:20:04.518 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:04.521+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.572 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.769390085, 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.573 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.574 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:20:04.575 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.576 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.576 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.576 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.577 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.578 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.74741693300001, 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.579 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:20:04.580 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:20:04.581 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.582 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.74696057700001, 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.583 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:20:04.584 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:20:04.585 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.586 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.587 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.727918821, 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.588 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:20:04.589 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.590 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.591 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.592 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.71962976200001, 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.593 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:20:04.594 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:20:04.595 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:20:04.596 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.71914175800001, 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.597 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.598 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.599 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.600 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:20:04.601 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.71866732500001, 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.602 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.603 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.604 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.605 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.71827395, 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.606 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.607 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.608 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.609 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:04.610 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.712087513, 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.611 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:20:04.612 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.613 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.614 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.70728406200001, 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.615 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.616 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:20:04.617 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:20:04.618 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.70470860399999, 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.619 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.620 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.621 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:20:04.623 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:20:04.624 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.701922908, 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:04.625 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.626 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.627 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.627 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:20:04.627 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:20:04.627 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.627 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.628 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:20:04.629 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:20:04.630 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.631 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.696920285, 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.632 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.633 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.634 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:20:04.635 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.636 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.637 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:20:04.638 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.52574422399999, 2024-05-08T17:20:04.639 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:04.640 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.641 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:20:04.642 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:20:04.643 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:20:04.644 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 210.11516025399999, 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.645 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.646 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.647 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:20:04.648 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.649 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.650 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.651 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:20:04.652 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 209.64236314300001, 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.653 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.654 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:20:04.655 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.656 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:20:04.657 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.658 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.659 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 209.641777577, 2024-05-08T17:20:04.660 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.661 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:20:04.662 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.663 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.664 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 209.64133817000001, 2024-05-08T17:20:04.665 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.666 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:20:04.667 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.668 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.669 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 209.64069018500001, 2024-05-08T17:20:04.670 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.671 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.672 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.673 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:20:04.674 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 209.635228458, 2024-05-08T17:20:04.675 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.676 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:20:04.677 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.678 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:20:04.679 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:04.680 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:04.682 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:20:04.682 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:04.684+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:04.783 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:04.785+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:04.789 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:04.789 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:04.789 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:04.789 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:04.798 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:20:04.878 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=false 2024-05-08T17:20:04.883 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:04.886+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:04.946 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:04.946 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:04.946 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:04.946 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:04.946 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:04.956 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:20:04.984 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:04.987+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:05.021 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:20:05.024 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:20:05.024 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:20:05.056 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:20:05.056 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:20:05.085 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:05.087+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:05.089 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:20:05.089 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:20:05.100 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.100 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.108 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:20:05.108 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:20:05.110 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:20:05.126 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:20:05.127 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:20:05.127 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:20:05.132 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is set 2024-05-08T17:20:05.153 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:20:05.153 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:20:05.154 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:20:05.157 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:20:05.157 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:20:05.199 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:05.202+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:05.235 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.235 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.244 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:20:05.287 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:05.289+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:05.388 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:05.390+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:05.389 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.389 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.398 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:20:05.402 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":42,"quorum":[0,1],"quorum_names":["a","b"],"quorum_leader_name":"a","quorum_age":37,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:20:05.488 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:05.491+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:05.504 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.504 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.514 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:20:05.589 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:05.591+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:05.620 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.620 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.630 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:20:05.689 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:05.692+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:05.775 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.775 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.784 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:20:05.790 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:05.793+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:05.890 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:05.890 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:05.891 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:05.893+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:05.899 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:20:05.992 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:05.994+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:06.015 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.015 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.024 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:20:06.093 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:06.096+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:06.130 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.130 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.140 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:20:06.194 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:06.197+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:06.244 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.244 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.253 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:20:06.295 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:06.298+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:06.395 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:06.398+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:06.398 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.398 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.407 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_blocked_ops 2024-05-08T17:20:06.496 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:06.499+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:06.511 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.511 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.520 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_historic_ops 2024-05-08T17:20:06.597 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:06.600+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:06.625 INFO:teuthology.orchestra.run.smithi183.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.625 INFO:teuthology.orchestra.run.smithi183.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.634 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:20:06.699 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:06.701+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:06.744 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.744 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.754 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:20:06.838 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:06.841+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:06.900 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:06.900 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:06.901 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:06.904+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:06.909 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:20:07.001 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:07.004+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:07.055 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:07.055 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:07.065 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:20:07.106 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:07.109+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:07.149 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:20:07.159 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'osd.*' injectargs --osd_enable_op_tracker=true 2024-05-08T17:20:07.215 INFO:teuthology.orchestra.run.smithi154.stdout:ERROR: (22) Invalid argument 2024-05-08T17:20:07.215 INFO:teuthology.orchestra.run.smithi154.stdout:op_tracker tracking is not enabled now, so no ops are tracked currently, even those get stuck. Please enable "osd_enable_op_tracker", and the tracker will start to track new ops received afterwards. 2024-05-08T17:20:07.216 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:07.218+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:07.226 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:20:07.305 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:07.308+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:07.322 INFO:teuthology.orchestra.run.smithi154.stdout:osd.0: {} 2024-05-08T17:20:07.322 INFO:teuthology.orchestra.run.smithi154.stderr:osd.0: osd_enable_op_tracker = '' 2024-05-08T17:20:07.341 INFO:teuthology.orchestra.run.smithi154.stdout:osd.1: {} 2024-05-08T17:20:07.342 INFO:teuthology.orchestra.run.smithi154.stderr:osd.1: osd_enable_op_tracker = '' 2024-05-08T17:20:07.359 INFO:teuthology.orchestra.run.smithi154.stdout:osd.2: {} 2024-05-08T17:20:07.359 INFO:teuthology.orchestra.run.smithi154.stderr:osd.2: osd_enable_op_tracker = '' 2024-05-08T17:20:07.374 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:07.374 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:07.374 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:07.374 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:07.375 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:07.376 INFO:teuthology.orchestra.run.smithi154.stdout:osd.3: {} 2024-05-08T17:20:07.377 INFO:teuthology.orchestra.run.smithi154.stderr:osd.3: osd_enable_op_tracker = '' 2024-05-08T17:20:07.384 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:20:07.394 INFO:teuthology.orchestra.run.smithi154.stdout:osd.4: {} 2024-05-08T17:20:07.395 INFO:teuthology.orchestra.run.smithi154.stderr:osd.4: osd_enable_op_tracker = '' 2024-05-08T17:20:07.395 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.5 2024-05-08T17:20:07.406 INFO:tasks.ceph.osd.3.smithi154.stderr:2024-05-08T17:20:07.408+0000 7f614be9d640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 3 (PID: 32455) UID: 0 2024-05-08T17:20:07.413 INFO:teuthology.orchestra.run.smithi154.stdout:osd.6: {} 2024-05-08T17:20:07.413 INFO:teuthology.orchestra.run.smithi154.stderr:osd.6: osd_enable_op_tracker = '' 2024-05-08T17:20:07.413 INFO:teuthology.orchestra.run.smithi154.stderr:Error ENXIO: problem getting command descriptions from osd.7 2024-05-08T17:20:07.417 DEBUG:teuthology.orchestra.run:got remote process result: 6 2024-05-08T17:20:07.417 INFO:tasks.thrashosds.thrasher:Failed to tell all osds, ignoring 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.492 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 139.213635977, 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.493 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.494 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:20:07.495 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:20:07.496 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:20:07.497 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 77.151632082999996, 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:07.498 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.499 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.500 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.501 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 17.150266980000001, 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000241578, 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:07.502 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 6 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:07.503 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345668+0000", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.858e-06 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345685+0000", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7263000000000001e-05 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345698+0000", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3237e-05 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:07.504 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345707+0000", 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3639999999999999e-06 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345789+0000", 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2301999999999999e-05 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345908+0000", 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118554 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.505 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.506 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.506 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.506 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:07.507 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:20:07.508 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:07.509+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:07.607 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:07.610+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:07.654 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:07.654 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:07.654 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:07.663 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:20:07.708 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:07.711+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:07.809 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:07.811+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:07.810 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:07.811 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:07.811 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:07.820 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:20:07.910 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:07.913+0000 7f0ca5c37640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 6 (PID: 32507) UID: 0 2024-05-08T17:20:07.966 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 139.68828733399999, 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:07.967 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:07.968 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:20:07.969 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:20:07.970 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:07.971 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:07.971 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:07.975 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:20:08.019 INFO:tasks.ceph.osd.1.smithi154.stderr:2024-05-08T17:20:08.022+0000 7ff98cb3f640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 1 (PID: 32446) UID: 0 2024-05-08T17:20:08.082 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:08.082 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:08.082 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:08.091 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:20:08.132 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:08.134+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:08.141 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:20:08.212 INFO:tasks.ceph.osd.4.smithi183.stderr:2024-05-08T17:20:08.215+0000 7f5a30e27640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 4 (PID: 32506) UID: 0 2024-05-08T17:20:08.238 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:08.239 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:08.239 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:08.247 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:20:08.313 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:08.316+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:08.353 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:08.354 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:08.354 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:08.354 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:20:08.354 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:08.362 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:20:08.415 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:08.418+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:08.417 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph quorum_status 2024-05-08T17:20:08.480 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:08.480 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:08.481 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:08.490 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:20:08.515 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:08.518+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:08.518 INFO:tasks.thrashosds.thrasher:in_osds: [1, 2, 3, 4, 5, 6, 7, 0] out_osds: [] dead_osds: [5, 7] live_osds: [0, 1, 2, 3, 4, 6] 2024-05-08T17:20:08.518 INFO:tasks.thrashosds.thrasher:choose_action: min_in 4 min_out 0 min_live 2 min_dead 0 chance_down 0.40 2024-05-08T17:20:08.518 INFO:tasks.thrashosds.thrasher:Killing osd 6, live_osds are [0, 1, 2, 3, 4, 6] 2024-05-08T17:20:08.518 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T17:20:08.518 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:20:08.564 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T17:20:08.564 INFO:tasks.thrashosds.thrasher:Testing ceph-objectstore-tool on down osd.6 2024-05-08T17:20:08.564 INFO:tasks.thrashosds.thrasher:remote for osd 6 is ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:20:08.564 INFO:tasks.thrashosds.thrasher:remote for exp osd 7 is ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:20:08.564 DEBUG:teuthology.orchestra.run.smithi183:> type ceph-objectstore-tool 2024-05-08T17:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:20:08.598 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:08.607 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:20:08.613 INFO:teuthology.orchestra.run.smithi183.stdout:ceph-objectstore-tool is /usr/bin/ceph-objectstore-tool 2024-05-08T17:20:08.613 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op list-pgs 2024-05-08T17:20:08.616 INFO:tasks.ceph.osd.2.smithi154.stderr:2024-05-08T17:20:08.619+0000 7f3924961640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 2 (PID: 32447) UID: 0 2024-05-08T17:20:08.717 INFO:tasks.ceph.osd.0.smithi154.stderr:2024-05-08T17:20:08.720+0000 7f78c2f17640 -1 received signal: Hangup from /usr/bin/python3 /bin/daemon-helper kill ceph-osd -f --cluster ceph -i 0 (PID: 32440) UID: 0 2024-05-08T17:20:08.760 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:08.760 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:20:08.760 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:20:08.760 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:20:08.760 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:08.769 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_ops_in_flight 2024-05-08T17:20:08.775 INFO:teuthology.orchestra.run.smithi154.stdout:{"election_epoch":44,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_leader_name":"a","quorum_age":0,"features":{"quorum_con":"4540701547738038271","quorum_mon":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"]},"monmap":{"epoch":1,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","modified":"2024-05-08T17:15:43.320397Z","created":"2024-05-08T17:15:43.320397Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":3,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3300","nonce":0},{"type":"v1","addr":"172.21.15.154:6789","nonce":0}]},"addr":"172.21.15.154:6789/0","public_addr":"172.21.15.154:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:3300","nonce":0},{"type":"v1","addr":"172.21.15.183:6789","nonce":0}]},"addr":"172.21.15.183:6789/0","public_addr":"172.21.15.183:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:3301","nonce":0},{"type":"v1","addr":"172.21.15.154:6790","nonce":0}]},"addr":"172.21.15.154:6790/0","public_addr":"172.21.15.154:6790/0","priority":0,"weight":0,"crush_location":"{}"}]}} 2024-05-08T17:20:08.789 INFO:tasks.mon_recovery.ceph_manager:quorum is size 3 2024-05-08T17:20:08.790 DEBUG:teuthology.run_tasks:Unwinding manager thrashosds 2024-05-08T17:20:08.800 INFO:tasks.thrashosds:joining thrashosds 2024-05-08T17:20:08.876 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:08.876 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:08.876 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:08.876 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:08.885 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_blocked_ops 2024-05-08T17:20:08.991 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:08.991 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:08.991 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.000 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.0.asok dump_historic_ops 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:1 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'1, mlcod=0'0)", 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.30402405699999, 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0068995200000000001, 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.107 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.108 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805269+0000", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0200000000000002e-07 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805275+0000", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.302e-06 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805284+0000", 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5790000000000004e-06 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.109 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805291+0000", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8839999999999997e-06 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805333+0000", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2215999999999997e-05 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.805356+0000", 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2487e-05 2024-05-08T17:20:09.110 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812121+0000", 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0067650849999999997 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.812168+0000", 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7364999999999998e-05 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.111 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:17 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'7, mlcod=15'5)", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.28205090500001, 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019595609999999999, 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.112 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827242+0000", 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.1800000000000005e-07 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827248+0000", 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.2510000000000002e-06 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.113 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827256+0000", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4360000000000002e-06 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827261+0000", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.0189999999999997e-06 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827305+0000", 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4165000000000002e-05 2024-05-08T17:20:09.114 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827320+0000", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.4304000000000001e-05 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829168+0000", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018483989999999999 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.829201+0000", 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3169e-05 2024-05-08T17:20:09.115 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:18 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'8, mlcod=15'5)", 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.281594549, 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041699140000000003, 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.116 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827698+0000", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4900000000000001e-07 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.117 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827703+0000", 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.143e-06 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827710+0000", 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3370000000000002e-06 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827714+0000", 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.1799999999999998e-06 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.118 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827749+0000", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5209000000000001e-05 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.827762+0000", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2969999999999999e-05 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831824+0000", 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.004061889 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.119 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.831868+0000", 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.3736999999999998e-05 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:24 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'14, mlcod=15'12)", 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.262552793, 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020386279999999998, 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.120 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846740+0000", 2024-05-08T17:20:09.121 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5599999999999995e-07 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846745+0000", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9529999999999998e-06 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846753+0000", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7659999999999998e-06 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846758+0000", 2024-05-08T17:20:09.122 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.639e-06 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846790+0000", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1857999999999999e-05 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.846801+0000", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1345999999999999e-05 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848752+0000", 2024-05-08T17:20:09.123 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00195111 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.848778+0000", 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6400000000000001e-05 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:34 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'20, mlcod=15'15)", 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.25426373400001, 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0031439440000000001, 2024-05-08T17:20:09.124 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.125 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855029+0000", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8299999999999998e-07 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855039+0000", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0114e-05 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855044+0000", 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.8230000000000002e-06 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.126 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855047+0000", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2880000000000001e-06 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855073+0000", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6010999999999999e-05 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855084+0000", 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.127 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858152+0000", 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030685830000000002 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858173+0000", 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0299999999999999e-05 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.128 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:35 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'21, mlcod=15'15)", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.25377573, 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028137290000000001, 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.129 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855517+0000", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4600000000000001e-07 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855573+0000", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5862999999999997e-05 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.130 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855579+0000", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.7049999999999996e-06 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855582+0000", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0589999999999998e-06 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857777+0000", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0021953200000000002 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.131 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.857789+0000", 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1816e-05 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858312+0000", 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000522936 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858331+0000", 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.8783999999999998e-05 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.132 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:36 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'22, mlcod=15'15)", 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.25330129700001, 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0025304160000000002, 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.133 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855991+0000", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855992+0000", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8700000000000002e-07 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855994+0000", 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5110000000000002e-06 2024-05-08T17:20:09.134 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.855999+0000", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9830000000000004e-06 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856003+0000", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5389999999999999e-06 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858019+0000", 2024-05-08T17:20:09.135 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002016636 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858036+0000", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.6546999999999999e-05 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858505+0000", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00046875900000000002 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858522+0000", 2024-05-08T17:20:09.136 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7153999999999999e-05 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:37 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'23, mlcod=15'15)", 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.25290792199999, 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0048989239999999998, 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.137 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856385+0000", 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9200000000000002e-07 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.138 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856387+0000", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4940000000000002e-06 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856393+0000", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.164e-06 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.856395+0000", 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7099999999999999e-06 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.139 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858197+0000", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018020950000000001 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.858209+0000", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1389999999999999e-05 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861256+0000", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.003047494 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.140 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.861284+0000", 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.7285e-05 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:39 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'25, mlcod=15'22)", 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.24672148499999, 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0039720110000000001, 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.141 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862571+0000", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862572+0000", 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.3099999999999998e-07 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.142 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862638+0000", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.6328999999999995e-05 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862646+0000", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6860000000000006e-06 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862650+0000", 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.07e-06 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.143 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862681+0000", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0845e-05 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.862693+0000", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2186e-05 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866513+0000", 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0038200249999999999 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.144 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.866543+0000", 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0338999999999999e-05 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:40 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'26, mlcod=15'24)", 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.24191803400001, 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019307739999999999, 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.145 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867375+0000", 2024-05-08T17:20:09.146 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.7100000000000001e-07 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867380+0000", 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4299999999999999e-06 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867389+0000", 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.9069999999999996e-06 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.147 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867394+0000", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5539999999999997e-06 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867433+0000", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8546e-05 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.867446+0000", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2853e-05 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.148 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869280+0000", 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018349110000000001 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869305+0000", 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.4902e-05 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:41 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'27, mlcod=15'25)", 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.23934257600001, 2024-05-08T17:20:09.149 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0020692729999999999, 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869950+0000", 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.150 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869951+0000", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5400000000000001e-07 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869955+0000", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.7999999999999998e-06 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869964+0000", 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.5259999999999993e-06 2024-05-08T17:20:09.151 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.869970+0000", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1569999999999997e-06 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870005+0000", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5305999999999997e-05 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.870018+0000", 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2566e-05 2024-05-08T17:20:09.152 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.871992+0000", 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0019744229999999999 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872019+0000", 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6940999999999998e-05 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.153 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:42 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'28, mlcod=15'26)", 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.23655687999999, 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00194539, 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.154 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872736+0000", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.4899999999999995e-07 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872741+0000", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4449999999999998e-06 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.155 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872750+0000", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.1179999999999998e-06 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872757+0000", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0099999999999998e-06 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872792+0000", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5500000000000002e-05 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.872804+0000", 2024-05-08T17:20:09.156 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2131e-05 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874659+0000", 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001854372 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.874681+0000", 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.2265000000000001e-05 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.157 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:48 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'30, mlcod=15'28)", 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.231554257, 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00204872, 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.158 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877738+0000", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877739+0000", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.5499999999999998e-07 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877743+0000", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.048e-06 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.159 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877750+0000", 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.3259999999999998e-06 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877756+0000", 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.5219999999999998e-06 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877787+0000", 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1576999999999999e-05 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.160 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.877799+0000", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1435e-05 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879762+0000", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001963041 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:33.879787+0000", 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.5316e-05 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.161 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:110 1.0 e15/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 15'72, mlcod=15'66)", 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 215.06037819599999, 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.001914708, 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.162 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048914+0000", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048915+0000", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0200000000000003e-07 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048920+0000", 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.339e-06 2024-05-08T17:20:09.163 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048928+0000", 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9000000000000006e-06 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048932+0000", 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4279999999999998e-06 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.164 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048969+0000", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.6736000000000002e-05 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.048982+0000", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3142000000000001e-05 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050785+0000", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0018025770000000001 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.165 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.050829+0000", 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.4184000000000003e-05 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:143 1.0 e15/14 1:30306672:devicehealth::main.db.0000000000000000:head v 15'101, mlcod=15'98)", 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.64979422600001, 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029579179999999999, 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.166 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459498+0000", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459499+0000", 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.7899999999999999e-07 2024-05-08T17:20:09.167 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459570+0000", 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1742999999999994e-05 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459578+0000", 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6760000000000007e-06 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.168 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459583+0000", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1510000000000003e-06 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459615+0000", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1902000000000001e-05 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.459627+0000", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2138e-05 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.169 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462427+0000", 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.002799447 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.462456+0000", 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9482e-05 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:156 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'110, mlcod=15'105)", 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:09.170 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.17699711500001, 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030177149999999998, 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.171 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932295+0000", 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932296+0000", 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.4799999999999999e-07 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932299+0000", 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.8710000000000001e-06 2024-05-08T17:20:09.172 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932305+0000", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.1709999999999999e-06 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932309+0000", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8029999999999998e-06 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932338+0000", 2024-05-08T17:20:09.173 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.9607e-05 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932349+0000", 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0542e-05 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935277+0000", 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0029284559999999999 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.174 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.935313+0000", 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.5917000000000002e-05 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:157 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'111, mlcod=16'107)", 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.17641154899999, 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0052079320000000002, 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.175 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932881+0000", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932882+0000", 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.51e-07 2024-05-08T17:20:09.176 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932888+0000", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.9800000000000003e-06 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932895+0000", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.7170000000000008e-06 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932901+0000", 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.0390000000000001e-06 2024-05-08T17:20:09.177 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932934+0000", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2499999999999997e-05 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.932945+0000", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0974e-05 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938056+0000", 2024-05-08T17:20:09.178 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0051107720000000004 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938089+0000", 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.3399000000000003e-05 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:158 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'112, mlcod=16'107)", 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.17597214200001, 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0047978609999999996, 2024-05-08T17:20:09.179 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933320+0000", 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.180 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933321+0000", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.2199999999999999e-07 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933370+0000", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.9490999999999998e-05 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933377+0000", 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0249999999999997e-06 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.181 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933383+0000", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1399999999999999e-06 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933414+0000", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.1353999999999998e-05 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933426+0000", 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.2153999999999999e-05 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.182 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938091+0000", 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0046652789999999996 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938118+0000", 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6996e-05 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.183 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:159 1.0 e16/14 1:1b0e96ae:devicehealth::main.db-journal.0000000000000000:head v 16'113, mlcod=16'108)", 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.17532415700001, 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0041798219999999997, 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.184 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933968+0000", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933969+0000", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 4.0400000000000002e-07 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933973+0000", 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8589999999999998e-06 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.185 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933981+0000", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.6249999999999996e-06 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.933986+0000", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1189999999999996e-06 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934017+0000", 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.0315999999999999e-05 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.186 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.934028+0000", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0971999999999999e-05 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938122+0000", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0040943350000000002 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.938148+0000", 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.6191999999999999e-05 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.187 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_repop(client.4215.0:161 1.0 e16/14 1:30306672:devicehealth::main.db.0000000000000000:head v 16'115, mlcod=16'112)", 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 214.16986242999999, 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0030093110000000002, 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "commit sent; apply or cleanup", 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:09.188 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939430+0000", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.2000000000000001e-07 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939505+0000", 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.4952999999999999e-05 2024-05-08T17:20:09.189 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939511+0000", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 5.1290000000000004e-06 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939514+0000", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.179e-06 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939577+0000", 2024-05-08T17:20:09.190 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3675000000000003e-05 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "started", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.939589+0000", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1705999999999999e-05 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "commit_sent", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942401+0000", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.0028120829999999999 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:16:34.942439+0000", 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 3.8266000000000003e-05 2024-05-08T17:20:09.191 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.192 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.192 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.192 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.192 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.192 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.193 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_ops_in_flight 2024-05-08T17:20:09.298 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.299 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:09.299 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:09.299 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.308 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_blocked_ops 2024-05-08T17:20:09.414 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.414 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:09.415 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:09.415 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:09.415 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.423 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.1.asok dump_historic_ops 2024-05-08T17:20:09.529 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.529 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:2 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 141.25098974299999, 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 2.0605912420000001, 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:09.530 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 2 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282297+0000", 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.531 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282299+0000", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.5030000000000001e-06 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282305+0000", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.4799999999999998e-06 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282314+0000", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.4970000000000001e-06 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.532 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282320+0000", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3990000000000001e-06 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.8569e-05 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for readable", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282451+0000", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.2069999999999994e-05 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.533 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339855+0000", 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.0574045679999999 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "waiting for peered", 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:49.339867+0000", 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1894e-05 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342817+0000", 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.002949624 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.534 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:50.342888+0000", 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.1637999999999994e-05 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:4 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e43)", 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 79.188985849000005, 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00025490099999999998, 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:09.535 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 4 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344301+0000", 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.536 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344302+0000", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.359e-06 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344310+0000", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.6629999999999993e-06 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344324+0000", 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3621000000000001e-05 2024-05-08T17:20:09.537 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344335+0000", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.1518000000000001e-05 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344448+0000", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000112688 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:18:50.344556+0000", 2024-05-08T17:20:09.538 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000108052 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:6 2.2 2:4e99cc3e:::rbd_mirror_snapshot_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e61)", 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 19.187620746, 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000241578, 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:09.539 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 6 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345666+0000", 2024-05-08T17:20:09.540 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345668+0000", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.858e-06 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345685+0000", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7263000000000001e-05 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345698+0000", 2024-05-08T17:20:09.541 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.3237e-05 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345707+0000", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.3639999999999999e-06 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345789+0000", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 8.2301999999999999e-05 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:19:50.345908+0000", 2024-05-08T17:20:09.542 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000118554 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.543 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.544 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_ops_in_flight 2024-05-08T17:20:09.650 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.650 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:09.650 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:09.650 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.659 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_blocked_ops 2024-05-08T17:20:09.743 INFO:teuthology.orchestra.run.smithi183.stdout:2.3 2024-05-08T17:20:09.743 INFO:teuthology.orchestra.run.smithi183.stdout:2.0 2024-05-08T17:20:09.743 INFO:teuthology.orchestra.run.smithi183.stdout:2.5 2024-05-08T17:20:09.743 INFO:teuthology.orchestra.run.smithi183.stdout:1.0 2024-05-08T17:20:09.765 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.765 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:09.765 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:09.766 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:09.766 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.774 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.2.asok dump_historic_ops 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [ 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "description": "osd_op(client.4178.0:1 2.1 2:88c1567c:::rbd_trash_purge_schedule:head [omap-get-vals] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e22)", 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "initiated_at": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:09.880 INFO:teuthology.orchestra.run.smithi154.stdout: "age": 141.60149696400001, 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.00020073000000000001, 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "continuous": false, 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "type_data": { 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "flag_point": "reached pg", 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "client_info": { 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "client": "client.4178", 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "client_addr": "172.21.15.154:0/1026403679", 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "tid": 1 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "events": [ 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "initiated", 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.881 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "header_read", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282293+0000", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "throttled", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282295+0000", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 1.7999999999999999e-06 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "all_read", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282301+0000", 2024-05-08T17:20:09.882 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 6.3069999999999999e-06 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "dispatched", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282311+0000", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 9.5740000000000002e-06 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "queued_for_pg", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282319+0000", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.9999999999999996e-06 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "reached_pg", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282389+0000", 2024-05-08T17:20:09.883 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 7.0046e-05 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: }, 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: { 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: "event": "done", 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: "time": "2024-05-08T17:17:48.282494+0000", 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 0.000105003 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: } 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout: ] 2024-05-08T17:20:09.884 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:09.888 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_ops_in_flight 2024-05-08T17:20:09.994 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:09.995 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:09.995 INFO:teuthology.orchestra.run.smithi154.stdout: "num_ops": 0 2024-05-08T17:20:09.995 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:10.003 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_blocked_ops 2024-05-08T17:20:10.109 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:10.109 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [], 2024-05-08T17:20:10.109 INFO:teuthology.orchestra.run.smithi154.stdout: "complaint_time": 30, 2024-05-08T17:20:10.109 INFO:teuthology.orchestra.run.smithi154.stdout: "num_blocked_ops": 0 2024-05-08T17:20:10.109 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:10.117 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.3.asok dump_historic_ops 2024-05-08T17:20:10.156 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset noscrub 2024-05-08T17:20:10.226 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:20:10.226 INFO:teuthology.orchestra.run.smithi154.stdout: "size": 20, 2024-05-08T17:20:10.226 INFO:teuthology.orchestra.run.smithi154.stdout: "duration": 600, 2024-05-08T17:20:10.226 INFO:teuthology.orchestra.run.smithi154.stdout: "ops": [] 2024-05-08T17:20:10.227 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:20:10.236 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_ops_in_flight 2024-05-08T17:20:10.271 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --op export --pgid 2.5 --file /var/log/ceph/exp.2.5.7 2024-05-08T17:20:10.346 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:10.346 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:10.346 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:10.346 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:10.355 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_blocked_ops 2024-05-08T17:20:10.483 INFO:teuthology.orchestra.run.smithi154.stderr:noscrub is unset 2024-05-08T17:20:10.497 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd unset nodeep-scrub 2024-05-08T17:20:10.501 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:10.501 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:10.501 INFO:teuthology.orchestra.run.smithi183.stdout: "complaint_time": 30, 2024-05-08T17:20:10.501 INFO:teuthology.orchestra.run.smithi183.stdout: "num_blocked_ops": 0 2024-05-08T17:20:10.501 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:10.510 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 30 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.4.asok dump_historic_ops 2024-05-08T17:20:10.614 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:10.614 INFO:teuthology.orchestra.run.smithi183.stdout: "size": 20, 2024-05-08T17:20:10.614 INFO:teuthology.orchestra.run.smithi183.stdout: "duration": 600, 2024-05-08T17:20:10.614 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [] 2024-05-08T17:20:10.614 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:11.407 INFO:teuthology.orchestra.run.smithi183.stderr:Exporting 2.5 info 2.5( empty local-lis/les=39/40 n=0 ec=15/15 lis/c=39/39 les/c/f=40/40/0 sis=39) 2024-05-08T17:20:11.408 INFO:teuthology.orchestra.run.smithi183.stderr:Export successful 2024-05-08T17:20:11.489 INFO:teuthology.orchestra.run.smithi154.stderr:nodeep-scrub is unset 2024-05-08T17:20:11.924 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-7 --journal-path /var/lib/ceph/osd/ceph-7/journal --force --op remove --pgid 2.5 2024-05-08T17:20:13.018 INFO:teuthology.orchestra.run.smithi183.stdout: marking collection for removal 2024-05-08T17:20:13.018 INFO:teuthology.orchestra.run.smithi183.stdout:setting '_remove' omap key 2024-05-08T17:20:13.018 INFO:teuthology.orchestra.run.smithi183.stdout:finish_remove_pgs 2.5_head removing 2.5 2024-05-08T17:20:13.019 INFO:teuthology.orchestra.run.smithi183.stdout:Remove successful 2024-05-08T17:20:13.534 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --no-mon-config '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal --op list-pgs 2024-05-08T17:20:14.641 INFO:teuthology.orchestra.run.smithi183.stdout:2.6 2024-05-08T17:20:15.171 INFO:tasks.thrashosds.thrasher:Moving pg 2.5 from osd.7 to osd.6 2024-05-08T17:20:15.172 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-objectstore-tool --err-to-stderr --data-path /var/lib/ceph/osd/ceph-6 --journal-path /var/lib/ceph/osd/ceph-6/journal '--log-file=/var/log/ceph/objectstore_tool.$pid.log' --op import --file /var/log/ceph/exp.2.5.7 2024-05-08T17:20:16.271 INFO:teuthology.orchestra.run.smithi183.stdout:get_pg_num_history pg_num_history pg_num_history(e71 pg_nums {1={14=1},2={15=8}} deleted_pools ) 2024-05-08T17:20:16.271 INFO:teuthology.orchestra.run.smithi183.stdout:Importing pgid 2.5 2024-05-08T17:20:16.271 INFO:teuthology.orchestra.run.smithi183.stdout:write_pg epoch 40 info 2.5( empty local-lis/les=39/40 n=0 ec=15/15 lis/c=39/39 les/c/f=40/40/0 sis=39) 2024-05-08T17:20:16.276 INFO:teuthology.orchestra.run.smithi183.stdout:Import successful 2024-05-08T17:20:16.792 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f /var/log/ceph/exp.2.5.7 2024-05-08T17:20:21.823 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-08T17:20:21.823 INFO:tasks.thrashosds.thrasher:Reviving osd 5 2024-05-08T17:20:21.823 INFO:tasks.ceph.osd.5:Restarting daemon 2024-05-08T17:20:21.824 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2024-05-08T17:20:21.827 INFO:tasks.ceph.osd.5:Started 2024-05-08T17:20:21.827 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:20:21.937 INFO:teuthology.orchestra.run.smithi183.stderr:no valid command found; 10 closest matches: 2024-05-08T17:20:21.937 INFO:teuthology.orchestra.run.smithi183.stderr:0 2024-05-08T17:20:21.937 INFO:teuthology.orchestra.run.smithi183.stderr:1 2024-05-08T17:20:21.937 INFO:teuthology.orchestra.run.smithi183.stderr:2 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:abort 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:assert 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:config diff 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:config diff get 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:config get 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:config help [] 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:config set ... 2024-05-08T17:20:21.938 INFO:teuthology.orchestra.run.smithi183.stderr:admin_socket: invalid command 2024-05-08T17:20:21.940 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T17:20:21.940 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-5, ['dump_ops_in_flight'] 2024-05-08T17:20:22.423 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:20:22.425+0000 7f280c611640 -1 Falling back to public interface 2024-05-08T17:20:25.868 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:20:25.871+0000 7f280c611640 -1 osd.5 22 log_to_monitors true 2024-05-08T17:20:26.848 INFO:tasks.ceph.osd.5.smithi183.stderr:2024-05-08T17:20:26.850+0000 7f2803d32640 -1 osd.5 76 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:20:26.941 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.5.asok dump_ops_in_flight 2024-05-08T17:20:27.058 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:27.058 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:27.058 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:27.068 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-08T17:20:27.068 INFO:tasks.thrashosds.thrasher:Reviving osd 6 2024-05-08T17:20:27.068 INFO:tasks.ceph.osd.6:Restarting daemon 2024-05-08T17:20:27.068 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2024-05-08T17:20:27.071 INFO:tasks.ceph.osd.6:Started 2024-05-08T17:20:27.071 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:20:27.171 INFO:teuthology.orchestra.run.smithi183.stderr:no valid command found; 10 closest matches: 2024-05-08T17:20:27.171 INFO:teuthology.orchestra.run.smithi183.stderr:0 2024-05-08T17:20:27.171 INFO:teuthology.orchestra.run.smithi183.stderr:1 2024-05-08T17:20:27.171 INFO:teuthology.orchestra.run.smithi183.stderr:2 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:abort 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:assert 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:config diff 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:config diff get 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:config get 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:config help [] 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:config set ... 2024-05-08T17:20:27.172 INFO:teuthology.orchestra.run.smithi183.stderr:admin_socket: invalid command 2024-05-08T17:20:27.173 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T17:20:27.174 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-6, ['dump_ops_in_flight'] 2024-05-08T17:20:27.657 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:27.660+0000 7f2b3f221640 -1 Falling back to public interface 2024-05-08T17:20:31.106 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:31.108+0000 7f2b3f221640 -1 osd.6 71 log_to_monitors true 2024-05-08T17:20:31.915 INFO:tasks.ceph.osd.6.smithi183.stderr:2024-05-08T17:20:31.918+0000 7f2b36ab0640 -1 osd.6 80 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:20:32.175 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.6.asok dump_ops_in_flight 2024-05-08T17:20:32.292 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:32.292 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:32.292 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:32.301 INFO:tasks.thrashosds.thrasher:reviving osd 2024-05-08T17:20:32.301 INFO:tasks.thrashosds.thrasher:Reviving osd 7 2024-05-08T17:20:32.302 INFO:tasks.ceph.osd.7:Restarting daemon 2024-05-08T17:20:32.302 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2024-05-08T17:20:32.304 INFO:tasks.ceph.osd.7:Started 2024-05-08T17:20:32.305 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:no valid command found; 10 closest matches: 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:0 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:1 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:2 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:abort 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:assert 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:config diff 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:config diff get 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:config get 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:config help [] 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:config set ... 2024-05-08T17:20:32.405 INFO:teuthology.orchestra.run.smithi183.stderr:admin_socket: invalid command 2024-05-08T17:20:32.407 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-05-08T17:20:32.407 INFO:tasks.ceph.ceph_manager.ceph:waiting on admin_socket for osd-7, ['dump_ops_in_flight'] 2024-05-08T17:20:32.892 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:20:32.894+0000 7f7a8d60e640 -1 Falling back to public interface 2024-05-08T17:20:36.346 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:20:36.348+0000 7f7a8d60e640 -1 osd.7 66 log_to_monitors true 2024-05-08T17:20:36.951 INFO:tasks.ceph.osd.7.smithi183.stderr:2024-05-08T17:20:36.954+0000 7f7a853cc640 -1 osd.7 84 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T17:20:37.408 DEBUG:teuthology.orchestra.run.smithi183:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 0 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-osd.7.asok dump_ops_in_flight 2024-05-08T17:20:37.526 INFO:teuthology.orchestra.run.smithi183.stdout:{ 2024-05-08T17:20:37.527 INFO:teuthology.orchestra.run.smithi183.stdout: "ops": [], 2024-05-08T17:20:37.527 INFO:teuthology.orchestra.run.smithi183.stdout: "num_ops": 0 2024-05-08T17:20:37.527 INFO:teuthology.orchestra.run.smithi183.stdout:} 2024-05-08T17:20:37.536 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -- tell 'mon.*' injectargs --mon_osd_down_out_interval=600 2024-05-08T17:20:37.661 INFO:teuthology.orchestra.run.smithi154.stdout:mon.a: {} 2024-05-08T17:20:37.661 INFO:teuthology.orchestra.run.smithi154.stderr:mon.a: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:20:37.674 INFO:teuthology.orchestra.run.smithi154.stdout:mon.b: {} 2024-05-08T17:20:37.674 INFO:teuthology.orchestra.run.smithi154.stderr:mon.b: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:20:37.679 INFO:teuthology.orchestra.run.smithi154.stdout:mon.c: {} 2024-05-08T17:20:37.680 INFO:teuthology.orchestra.run.smithi154.stderr:mon.c: mon_osd_down_out_interval = '' (not observed, change may require restart) 2024-05-08T17:20:37.690 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 0 1 2024-05-08T17:20:38.944 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.0 to 1 (10000) 2024-05-08T17:20:38.961 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 0 1 2024-05-08T17:20:39.953 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.0 primary-affinity to 1 (8655362) 2024-05-08T17:20:39.969 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 1 1 2024-05-08T17:20:40.964 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.1 to 1 (10000) 2024-05-08T17:20:40.980 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 1 1 2024-05-08T17:20:41.970 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.1 primary-affinity to 1 (8655362) 2024-05-08T17:20:41.982 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 2 1 2024-05-08T17:20:42.974 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.2 to 1 (10000) 2024-05-08T17:20:42.989 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 2 1 2024-05-08T17:20:43.976 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.2 primary-affinity to 1 (8655362) 2024-05-08T17:20:43.993 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 3 1 2024-05-08T17:20:44.814 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.3 to 1 (10000) 2024-05-08T17:20:44.829 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 3 1 2024-05-08T17:20:45.991 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.3 primary-affinity to 1 (8655362) 2024-05-08T17:20:46.008 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 4 1 2024-05-08T17:20:46.993 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.4 to 1 (10000) 2024-05-08T17:20:47.010 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 4 1 2024-05-08T17:20:48.003 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.4 primary-affinity to 1 (8655362) 2024-05-08T17:20:48.018 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 5 1 2024-05-08T17:20:49.013 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.5 to 1 (10000) 2024-05-08T17:20:49.028 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 5 1 2024-05-08T17:20:50.015 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.5 primary-affinity to 1 (8655362) 2024-05-08T17:20:50.033 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 6 1 2024-05-08T17:20:51.029 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.6 to 1 (10000) 2024-05-08T17:20:51.044 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 6 1 2024-05-08T17:20:52.028 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.6 primary-affinity to 1 (8655362) 2024-05-08T17:20:52.045 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd reweight 7 1 2024-05-08T17:20:53.042 INFO:teuthology.orchestra.run.smithi154.stderr:reweighted osd.7 to 1 (10000) 2024-05-08T17:20:53.059 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd primary-affinity 7 1 2024-05-08T17:20:54.048 INFO:teuthology.orchestra.run.smithi154.stderr:set osd.7 primary-affinity to 1 (8655362) 2024-05-08T17:20:54.065 INFO:tasks.thrashosds.thrasher:joining the do_sighup greenlet 2024-05-08T17:20:54.065 INFO:tasks.thrashosds.thrasher:joining the do_optrack_toggle greenlet 2024-05-08T17:20:54.065 INFO:tasks.thrashosds.thrasher:joining the do_dump_ops greenlet 2024-05-08T17:20:54.065 INFO:tasks.thrashosds.thrasher:joining the do_noscrub_toggle greenlet 2024-05-08T17:20:54.066 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-05-08T17:20:54.066 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:20:54.352 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:54.352 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":101,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:20:54.051022+0000","last_up_change":"2024-05-08T17:20:37.933075+0000","last_in_change":"2024-05-08T17:18:41.921552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":22,"up_from":77,"up_thru":97,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":40330014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":40330014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":40330014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":40330014}]},"public_addr":"172.21.15.183:6800/40330014","cluster_addr":"172.21.15.183:6801/40330014","heartbeat_back_addr":"172.21.15.183:6803/40330014","heartbeat_front_addr":"172.21.15.183:6802/40330014","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":71,"up_from":81,"up_thru":15,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2007232062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":2007232062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":2007232062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2007232062}]},"public_addr":"172.21.15.183:6804/2007232062","cluster_addr":"172.21.15.183:6805/2007232062","heartbeat_back_addr":"172.21.15.183:6807/2007232062","heartbeat_front_addr":"172.21.15.183:6806/2007232062","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":66,"up_from":85,"up_thru":86,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2803215270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2803215270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2803215270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2803215270}]},"public_addr":"172.21.15.183:6808/2803215270","cluster_addr":"172.21.15.183:6809/2803215270","heartbeat_back_addr":"172.21.15.183:6811/2803215270","heartbeat_front_addr":"172.21.15.183:6810/2803215270","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":24},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":73},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":67}],"pg_upmap":[{"pgid":"2.7","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:20:54.366 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-05-08T17:20:54.366 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:20:54.654 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:54.654 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":101,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:20:54.051022+0000","last_up_change":"2024-05-08T17:20:37.933075+0000","last_in_change":"2024-05-08T17:18:41.921552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":22,"up_from":77,"up_thru":97,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":40330014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":40330014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":40330014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":40330014}]},"public_addr":"172.21.15.183:6800/40330014","cluster_addr":"172.21.15.183:6801/40330014","heartbeat_back_addr":"172.21.15.183:6803/40330014","heartbeat_front_addr":"172.21.15.183:6802/40330014","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":71,"up_from":81,"up_thru":15,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2007232062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":2007232062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":2007232062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2007232062}]},"public_addr":"172.21.15.183:6804/2007232062","cluster_addr":"172.21.15.183:6805/2007232062","heartbeat_back_addr":"172.21.15.183:6807/2007232062","heartbeat_front_addr":"172.21.15.183:6806/2007232062","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":66,"up_from":85,"up_thru":86,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2803215270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2803215270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2803215270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2803215270}]},"public_addr":"172.21.15.183:6808/2803215270","cluster_addr":"172.21.15.183:6809/2803215270","heartbeat_back_addr":"172.21.15.183:6811/2803215270","heartbeat_front_addr":"172.21.15.183:6810/2803215270","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":24},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":73},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":67}],"pg_upmap":[{"pgid":"2.7","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:20:54.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-05-08T17:20:54.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-05-08T17:20:54.668 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-05-08T17:20:54.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-05-08T17:20:54.669 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2024-05-08T17:20:54.670 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2024-05-08T17:20:54.670 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2024-05-08T17:20:54.673 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2024-05-08T17:20:54.919 INFO:teuthology.orchestra.run.smithi154.stdout:60129542203 2024-05-08T17:20:54.919 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T17:20:54.923 INFO:teuthology.orchestra.run.smithi154.stdout:330712481799 2024-05-08T17:20:54.923 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T17:20:54.924 INFO:teuthology.orchestra.run.smithi154.stdout:347892350982 2024-05-08T17:20:54.924 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T17:20:54.925 INFO:teuthology.orchestra.run.smithi154.stdout:60129542200 2024-05-08T17:20:54.925 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T17:20:54.926 INFO:teuthology.orchestra.run.smithi154.stdout:55834574903 2024-05-08T17:20:54.926 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T17:20:54.927 INFO:teuthology.orchestra.run.smithi154.stdout:60129542200 2024-05-08T17:20:54.928 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T17:20:54.930 INFO:teuthology.orchestra.run.smithi154.stdout:365072220166 2024-05-08T17:20:54.930 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T17:20:54.935 INFO:teuthology.orchestra.run.smithi154.stdout:55834574905 2024-05-08T17:20:54.935 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T17:20:55.513 INFO:teuthology.orchestra.run.smithi154.stdout:347892350981 2024-05-08T17:20:55.533 INFO:teuthology.orchestra.run.smithi154.stdout:60129542199 2024-05-08T17:20:55.534 INFO:tasks.ceph.ceph_manager.ceph:need seq 347892350982 got 347892350981 for osd.6 2024-05-08T17:20:55.539 INFO:teuthology.orchestra.run.smithi154.stdout:330712481798 2024-05-08T17:20:55.541 INFO:teuthology.orchestra.run.smithi154.stdout:55834574902 2024-05-08T17:20:55.548 INFO:teuthology.orchestra.run.smithi154.stdout:60129542199 2024-05-08T17:20:55.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542199 for osd.0 2024-05-08T17:20:55.556 INFO:teuthology.orchestra.run.smithi154.stdout:60129542202 2024-05-08T17:20:55.559 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574902 for osd.2 2024-05-08T17:20:55.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 330712481799 got 330712481798 for osd.5 2024-05-08T17:20:55.568 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542199 for osd.1 2024-05-08T17:20:55.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542202 for osd.3 2024-05-08T17:20:55.586 INFO:teuthology.orchestra.run.smithi154.stdout:365072220165 2024-05-08T17:20:55.591 INFO:teuthology.orchestra.run.smithi154.stdout:55834574904 2024-05-08T17:20:55.602 INFO:tasks.ceph.ceph_manager.ceph:need seq 365072220166 got 365072220165 for osd.7 2024-05-08T17:20:55.605 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574905 got 55834574904 for osd.4 2024-05-08T17:20:56.535 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2024-05-08T17:20:56.557 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-05-08T17:20:56.559 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-05-08T17:20:56.563 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2024-05-08T17:20:56.568 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-05-08T17:20:56.573 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-05-08T17:20:56.603 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2024-05-08T17:20:56.606 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2024-05-08T17:20:57.116 INFO:teuthology.orchestra.run.smithi154.stdout:347892350982 2024-05-08T17:20:57.137 INFO:tasks.ceph.ceph_manager.ceph:need seq 347892350982 got 347892350982 for osd.6 2024-05-08T17:20:57.137 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.154 INFO:teuthology.orchestra.run.smithi154.stdout:60129542203 2024-05-08T17:20:57.168 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542203 got 60129542203 for osd.3 2024-05-08T17:20:57.168 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.181 INFO:teuthology.orchestra.run.smithi154.stdout:55834574903 2024-05-08T17:20:57.189 INFO:teuthology.orchestra.run.smithi154.stdout:60129542200 2024-05-08T17:20:57.192 INFO:teuthology.orchestra.run.smithi154.stdout:330712481799 2024-05-08T17:20:57.200 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574903 got 55834574903 for osd.2 2024-05-08T17:20:57.200 INFO:teuthology.orchestra.run.smithi154.stdout:60129542200 2024-05-08T17:20:57.201 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.210 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542200 for osd.1 2024-05-08T17:20:57.211 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.212 INFO:tasks.ceph.ceph_manager.ceph:need seq 330712481799 got 330712481799 for osd.5 2024-05-08T17:20:57.212 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.217 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542200 got 60129542200 for osd.0 2024-05-08T17:20:57.217 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.222 INFO:teuthology.orchestra.run.smithi154.stdout:55834574905 2024-05-08T17:20:57.226 INFO:teuthology.orchestra.run.smithi154.stdout:365072220166 2024-05-08T17:20:57.236 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574905 got 55834574905 for osd.4 2024-05-08T17:20:57.236 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.239 INFO:tasks.ceph.ceph_manager.ceph:need seq 365072220166 got 365072220166 for osd.7 2024-05-08T17:20:57.239 DEBUG:teuthology.parallel:result is None 2024-05-08T17:20:57.240 INFO:tasks.ceph.ceph_manager.ceph:waiting for recovery to complete 2024-05-08T17:20:57.240 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:20:57.565 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:57.566 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:20:57.578 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-05-08T17:20:56.751404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227540,"kb_used_data":8864,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747180,"statfs":{"total":773094113280,"available":772861112320,"internally_reserved":0,"allocated":9076736,"data_stored":4255188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.716275"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579632+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:20:54.579632+0000","last_peered":"2024-05-08T17:20:54.579632+0000","last_clean":"2024-05-08T17:20:54.579632+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:20:54.579632+0000","last_undegraded":"2024-05-08T17:20:54.579632+0000","last_fullsized":"2024-05-08T17:20:54.579632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579401+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:54.579401+0000","last_peered":"2024-05-08T17:20:54.579401+0000","last_clean":"2024-05-08T17:20:54.579401+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:54.579401+0000","last_undegraded":"2024-05-08T17:20:54.579401+0000","last_fullsized":"2024-05-08T17:20:54.579401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578864+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:54.578864+0000","last_peered":"2024-05-08T17:20:54.578864+0000","last_clean":"2024-05-08T17:20:54.578864+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:54.578864+0000","last_undegraded":"2024-05-08T17:20:54.578864+0000","last_fullsized":"2024-05-08T17:20:54.578864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579385+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:54.579385+0000","last_peered":"2024-05-08T17:20:54.579385+0000","last_clean":"2024-05-08T17:20:54.579385+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:54.579385+0000","last_undegraded":"2024-05-08T17:20:54.579385+0000","last_fullsized":"2024-05-08T17:20:54.579385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054138+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:54.054138+0000","last_peered":"2024-05-08T17:20:54.054138+0000","last_clean":"2024-05-08T17:20:54.054138+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:54.054138+0000","last_undegraded":"2024-05-08T17:20:54.054138+0000","last_fullsized":"2024-05-08T17:20:54.054138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":180,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579619+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:20:54.579619+0000","last_peered":"2024-05-08T17:20:54.579619+0000","last_clean":"2024-05-08T17:20:54.579619+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:20:54.579619+0000","last_undegraded":"2024-05-08T17:20:54.579619+0000","last_fullsized":"2024-05-08T17:20:54.579619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578859+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:54.578859+0000","last_peered":"2024-05-08T17:20:54.578859+0000","last_clean":"2024-05-08T17:20:54.578859+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:54.578859+0000","last_undegraded":"2024-05-08T17:20:54.578859+0000","last_fullsized":"2024-05-08T17:20:54.578859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054154+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:54.054154+0000","last_peered":"2024-05-08T17:20:54.054154+0000","last_clean":"2024-05-08T17:20:54.054154+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:54.054154+0000","last_undegraded":"2024-05-08T17:20:54.054154+0000","last_fullsized":"2024-05-08T17:20:54.054154+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578846+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:54.578846+0000","last_peered":"2024-05-08T17:20:54.578846+0000","last_clean":"2024-05-08T17:20:54.578846+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:54.578846+0000","last_undegraded":"2024-05-08T17:20:54.578846+0000","last_fullsized":"2024-05-08T17:20:54.578846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343316,"statfs":{"total":96636764160,"available":96607555584,"internally_reserved":0,"allocated":1540096,"data_stored":930775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220166,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29224,"kb_used_data":1492,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342616,"statfs":{"total":96636764160,"available":96606838784,"internally_reserved":0,"allocated":1527808,"data_stored":926416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28812,"kb_used_data":960,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94343028,"statfs":{"total":96636764160,"available":96607260672,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481799,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28964,"kb_used_data":984,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342876,"statfs":{"total":96636764160,"available":96607105024,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:20:57.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:20:57.863 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:57.864 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:20:57.876 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-05-08T17:20:56.751404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227540,"kb_used_data":8864,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747180,"statfs":{"total":773094113280,"available":772861112320,"internally_reserved":0,"allocated":9076736,"data_stored":4255188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.716275"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579632+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:20:54.579632+0000","last_peered":"2024-05-08T17:20:54.579632+0000","last_clean":"2024-05-08T17:20:54.579632+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:20:54.579632+0000","last_undegraded":"2024-05-08T17:20:54.579632+0000","last_fullsized":"2024-05-08T17:20:54.579632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579401+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:54.579401+0000","last_peered":"2024-05-08T17:20:54.579401+0000","last_clean":"2024-05-08T17:20:54.579401+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:54.579401+0000","last_undegraded":"2024-05-08T17:20:54.579401+0000","last_fullsized":"2024-05-08T17:20:54.579401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578864+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:54.578864+0000","last_peered":"2024-05-08T17:20:54.578864+0000","last_clean":"2024-05-08T17:20:54.578864+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:54.578864+0000","last_undegraded":"2024-05-08T17:20:54.578864+0000","last_fullsized":"2024-05-08T17:20:54.578864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579385+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:54.579385+0000","last_peered":"2024-05-08T17:20:54.579385+0000","last_clean":"2024-05-08T17:20:54.579385+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:54.579385+0000","last_undegraded":"2024-05-08T17:20:54.579385+0000","last_fullsized":"2024-05-08T17:20:54.579385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054138+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:54.054138+0000","last_peered":"2024-05-08T17:20:54.054138+0000","last_clean":"2024-05-08T17:20:54.054138+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:54.054138+0000","last_undegraded":"2024-05-08T17:20:54.054138+0000","last_fullsized":"2024-05-08T17:20:54.054138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":180,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579619+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:20:54.579619+0000","last_peered":"2024-05-08T17:20:54.579619+0000","last_clean":"2024-05-08T17:20:54.579619+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:20:54.579619+0000","last_undegraded":"2024-05-08T17:20:54.579619+0000","last_fullsized":"2024-05-08T17:20:54.579619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578859+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:54.578859+0000","last_peered":"2024-05-08T17:20:54.578859+0000","last_clean":"2024-05-08T17:20:54.578859+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:54.578859+0000","last_undegraded":"2024-05-08T17:20:54.578859+0000","last_fullsized":"2024-05-08T17:20:54.578859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054154+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:54.054154+0000","last_peered":"2024-05-08T17:20:54.054154+0000","last_clean":"2024-05-08T17:20:54.054154+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:54.054154+0000","last_undegraded":"2024-05-08T17:20:54.054154+0000","last_fullsized":"2024-05-08T17:20:54.054154+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578846+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:54.578846+0000","last_peered":"2024-05-08T17:20:54.578846+0000","last_clean":"2024-05-08T17:20:54.578846+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:54.578846+0000","last_undegraded":"2024-05-08T17:20:54.578846+0000","last_fullsized":"2024-05-08T17:20:54.578846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343316,"statfs":{"total":96636764160,"available":96607555584,"internally_reserved":0,"allocated":1540096,"data_stored":930775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220166,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29224,"kb_used_data":1492,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342616,"statfs":{"total":96636764160,"available":96606838784,"internally_reserved":0,"allocated":1527808,"data_stored":926416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28812,"kb_used_data":960,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94343028,"statfs":{"total":96636764160,"available":96607260672,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481799,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28964,"kb_used_data":984,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342876,"statfs":{"total":96636764160,"available":96607105024,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:20:57.877 INFO:tasks.ceph.ceph_manager.ceph:recovered! 2024-05-08T17:20:57.877 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr_debug_aggressive_pg_num_changes 2024-05-08T17:20:58.186 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-05-08T17:20:58.195 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-05-08T17:20:58.196 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:20:58.479 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:58.480 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:20:58.492 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":227,"stamp":"2024-05-08T17:20:56.751404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227540,"kb_used_data":8864,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747180,"statfs":{"total":773094113280,"available":772861112320,"internally_reserved":0,"allocated":9076736,"data_stored":4255188,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.716275"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":120,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579632+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:20:54.579632+0000","last_peered":"2024-05-08T17:20:54.579632+0000","last_clean":"2024-05-08T17:20:54.579632+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:20:54.579632+0000","last_undegraded":"2024-05-08T17:20:54.579632+0000","last_fullsized":"2024-05-08T17:20:54.579632+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":184,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579401+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:54.579401+0000","last_peered":"2024-05-08T17:20:54.579401+0000","last_clean":"2024-05-08T17:20:54.579401+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:54.579401+0000","last_undegraded":"2024-05-08T17:20:54.579401+0000","last_fullsized":"2024-05-08T17:20:54.579401+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":37,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578864+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:54.578864+0000","last_peered":"2024-05-08T17:20:54.578864+0000","last_clean":"2024-05-08T17:20:54.578864+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:54.578864+0000","last_undegraded":"2024-05-08T17:20:54.578864+0000","last_fullsized":"2024-05-08T17:20:54.578864+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":187,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579385+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:54.579385+0000","last_peered":"2024-05-08T17:20:54.579385+0000","last_clean":"2024-05-08T17:20:54.579385+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:54.579385+0000","last_undegraded":"2024-05-08T17:20:54.579385+0000","last_fullsized":"2024-05-08T17:20:54.579385+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":141,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054138+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:54.054138+0000","last_peered":"2024-05-08T17:20:54.054138+0000","last_clean":"2024-05-08T17:20:54.054138+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:54.054138+0000","last_undegraded":"2024-05-08T17:20:54.054138+0000","last_fullsized":"2024-05-08T17:20:54.054138+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":180,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.579619+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:20:54.579619+0000","last_peered":"2024-05-08T17:20:54.579619+0000","last_clean":"2024-05-08T17:20:54.579619+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:20:54.579619+0000","last_undegraded":"2024-05-08T17:20:54.579619+0000","last_fullsized":"2024-05-08T17:20:54.579619+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":133,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578859+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:54.578859+0000","last_peered":"2024-05-08T17:20:54.578859+0000","last_clean":"2024-05-08T17:20:54.578859+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:54.578859+0000","last_undegraded":"2024-05-08T17:20:54.578859+0000","last_fullsized":"2024-05-08T17:20:54.578859+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":128,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.054154+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:54.054154+0000","last_peered":"2024-05-08T17:20:54.054154+0000","last_clean":"2024-05-08T17:20:54.054154+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:54.054154+0000","last_undegraded":"2024-05-08T17:20:54.054154+0000","last_fullsized":"2024-05-08T17:20:54.054154+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":283,"reported_epoch":101,"state":"active+clean","last_fresh":"2024-05-08T17:20:54.578846+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:54.578846+0000","last_peered":"2024-05-08T17:20:54.578846+0000","last_clean":"2024-05-08T17:20:54.578846+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:54.578846+0000","last_undegraded":"2024-05-08T17:20:54.578846+0000","last_fullsized":"2024-05-08T17:20:54.578846+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542200,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27984,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343856,"statfs":{"total":96636764160,"available":96608108544,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542200,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28524,"kb_used_data":1504,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343316,"statfs":{"total":96636764160,"available":96607555584,"internally_reserved":0,"allocated":1540096,"data_stored":930775,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220166,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29224,"kb_used_data":1492,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342616,"statfs":{"total":96636764160,"available":96606838784,"internally_reserved":0,"allocated":1527808,"data_stored":926416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350982,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28812,"kb_used_data":960,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94343028,"statfs":{"total":96636764160,"available":96607260672,"internally_reserved":0,"allocated":983040,"data_stored":393129,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481799,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28964,"kb_used_data":984,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342876,"statfs":{"total":96636764160,"available":96607105024,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574905,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574903,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28004,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343836,"statfs":{"total":96636764160,"available":96608088064,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:20:58.493 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:20:58.776 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:58.776 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:20:58.788 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-05-08T17:20:58.751766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227680,"kb_used_data":8996,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747040,"statfs":{"total":773094113280,"available":772860968960,"internally_reserved":0,"allocated":9211904,"data_stored":4303089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.742942+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:20:55.742942+0000","last_peered":"2024-05-08T17:20:55.742942+0000","last_clean":"2024-05-08T17:20:55.742942+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:20:55.742942+0000","last_undegraded":"2024-05-08T17:20:55.742942+0000","last_fullsized":"2024-05-08T17:20:55.742942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442459+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:55.442459+0000","last_peered":"2024-05-08T17:20:55.442459+0000","last_clean":"2024-05-08T17:20:55.442459+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:55.442459+0000","last_undegraded":"2024-05-08T17:20:55.442459+0000","last_fullsized":"2024-05-08T17:20:55.442459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153844+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:55.153844+0000","last_peered":"2024-05-08T17:20:55.153844+0000","last_clean":"2024-05-08T17:20:55.153844+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:55.153844+0000","last_undegraded":"2024-05-08T17:20:55.153844+0000","last_fullsized":"2024-05-08T17:20:55.153844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442760+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:55.442760+0000","last_peered":"2024-05-08T17:20:55.442760+0000","last_clean":"2024-05-08T17:20:55.442760+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:55.442760+0000","last_undegraded":"2024-05-08T17:20:55.442760+0000","last_fullsized":"2024-05-08T17:20:55.442760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743094+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:55.743094+0000","last_peered":"2024-05-08T17:20:55.743094+0000","last_clean":"2024-05-08T17:20:55.743094+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:55.743094+0000","last_undegraded":"2024-05-08T17:20:55.743094+0000","last_fullsized":"2024-05-08T17:20:55.743094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.742930+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:20:55.742930+0000","last_peered":"2024-05-08T17:20:55.742930+0000","last_clean":"2024-05-08T17:20:55.742930+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:20:55.742930+0000","last_undegraded":"2024-05-08T17:20:55.742930+0000","last_fullsized":"2024-05-08T17:20:55.742930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153939+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:55.153939+0000","last_peered":"2024-05-08T17:20:55.153939+0000","last_clean":"2024-05-08T17:20:55.153939+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:55.153939+0000","last_undegraded":"2024-05-08T17:20:55.153939+0000","last_fullsized":"2024-05-08T17:20:55.153939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743101+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:55.743101+0000","last_peered":"2024-05-08T17:20:55.743101+0000","last_clean":"2024-05-08T17:20:55.743101+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:55.743101+0000","last_undegraded":"2024-05-08T17:20:55.743101+0000","last_fullsized":"2024-05-08T17:20:55.743101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153794+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:55.153794+0000","last_peered":"2024-05-08T17:20:55.153794+0000","last_clean":"2024-05-08T17:20:55.153794+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:55.153794+0000","last_undegraded":"2024-05-08T17:20:55.153794+0000","last_fullsized":"2024-05-08T17:20:55.153794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29244,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342596,"statfs":{"total":96636764160,"available":96606818304,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481800,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28980,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342860,"statfs":{"total":96636764160,"available":96607088640,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:20:58.789 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-05-08T17:20:58.789 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:20:59.073 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:59.074 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:20:59.086 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":228,"stamp":"2024-05-08T17:20:58.751766+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227680,"kb_used_data":8996,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747040,"statfs":{"total":773094113280,"available":772860968960,"internally_reserved":0,"allocated":9211904,"data_stored":4303089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.000980"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":122,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.742942+0000","last_change":"2024-05-08T17:18:50.419247+0000","last_active":"2024-05-08T17:20:55.742942+0000","last_peered":"2024-05-08T17:20:55.742942+0000","last_clean":"2024-05-08T17:20:55.742942+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:20:55.742942+0000","last_undegraded":"2024-05-08T17:20:55.742942+0000","last_fullsized":"2024-05-08T17:20:55.742942+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:01:49.619886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442459+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:55.442459+0000","last_peered":"2024-05-08T17:20:55.442459+0000","last_clean":"2024-05-08T17:20:55.442459+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:55.442459+0000","last_undegraded":"2024-05-08T17:20:55.442459+0000","last_fullsized":"2024-05-08T17:20:55.442459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153844+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:55.153844+0000","last_peered":"2024-05-08T17:20:55.153844+0000","last_clean":"2024-05-08T17:20:55.153844+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:55.153844+0000","last_undegraded":"2024-05-08T17:20:55.153844+0000","last_fullsized":"2024-05-08T17:20:55.153844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442760+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:55.442760+0000","last_peered":"2024-05-08T17:20:55.442760+0000","last_clean":"2024-05-08T17:20:55.442760+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:55.442760+0000","last_undegraded":"2024-05-08T17:20:55.442760+0000","last_fullsized":"2024-05-08T17:20:55.442760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743094+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:55.743094+0000","last_peered":"2024-05-08T17:20:55.743094+0000","last_clean":"2024-05-08T17:20:55.743094+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:55.743094+0000","last_undegraded":"2024-05-08T17:20:55.743094+0000","last_fullsized":"2024-05-08T17:20:55.743094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":182,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.742930+0000","last_change":"2024-05-08T17:16:34.858982+0000","last_active":"2024-05-08T17:20:55.742930+0000","last_peered":"2024-05-08T17:20:55.742930+0000","last_clean":"2024-05-08T17:20:55.742930+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:20:55.742930+0000","last_undegraded":"2024-05-08T17:20:55.742930+0000","last_fullsized":"2024-05-08T17:20:55.742930+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T05:07:45.702829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153939+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:55.153939+0000","last_peered":"2024-05-08T17:20:55.153939+0000","last_clean":"2024-05-08T17:20:55.153939+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:55.153939+0000","last_undegraded":"2024-05-08T17:20:55.153939+0000","last_fullsized":"2024-05-08T17:20:55.153939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743101+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:55.743101+0000","last_peered":"2024-05-08T17:20:55.743101+0000","last_clean":"2024-05-08T17:20:55.743101+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:55.743101+0000","last_undegraded":"2024-05-08T17:20:55.743101+0000","last_fullsized":"2024-05-08T17:20:55.743101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153794+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:55.153794+0000","last_peered":"2024-05-08T17:20:55.153794+0000","last_clean":"2024-05-08T17:20:55.153794+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:55.153794+0000","last_undegraded":"2024-05-08T17:20:55.153794+0000","last_fullsized":"2024-05-08T17:20:55.153794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542203,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28008,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343832,"statfs":{"total":96636764160,"available":96608083968,"internally_reserved":0,"allocated":1007616,"data_stored":401847,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220167,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29244,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342596,"statfs":{"total":96636764160,"available":96606818304,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350983,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481800,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28980,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342860,"statfs":{"total":96636764160,"available":96607088640,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574906,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574904,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:20:59.087 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T17:20:59.373 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:20:59.373 INFO:teuthology.orchestra.run.smithi154.stdout:{"epoch":102,"fsid":"c69ba297-505d-4f4e-b390-bd22ef533cfc","created":"2024-05-08T17:16:22.335952+0000","modified":"2024-05-08T17:20:54.815808+0000","last_up_change":"2024-05-08T17:20:37.933075+0000","last_in_change":"2024-05-08T17:18:41.921552+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":7,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T17:16:32.734319+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-05-08T17:16:33.268021+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.9900000095367432,"score_stable":1.9900000095367432,"optimal_score":0.87999999523162842,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c1fd5e13-93ab-4b5a-91bd-08ed1a1e4858","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":85,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6808","nonce":3590430155}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6809","nonce":3590430155}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6811","nonce":3590430155}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6810","nonce":3590430155}]},"public_addr":"172.21.15.154:6808/3590430155","cluster_addr":"172.21.15.154:6809/3590430155","heartbeat_back_addr":"172.21.15.154:6811/3590430155","heartbeat_front_addr":"172.21.15.154:6810/3590430155","state":["exists","up"]},{"osd":1,"uuid":"7e66ef3b-5581-4f8c-9807-e16ea4ec972e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":81,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6812","nonce":2114037860}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6813","nonce":2114037860}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6815","nonce":2114037860}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6814","nonce":2114037860}]},"public_addr":"172.21.15.154:6812/2114037860","cluster_addr":"172.21.15.154:6813/2114037860","heartbeat_back_addr":"172.21.15.154:6815/2114037860","heartbeat_front_addr":"172.21.15.154:6814/2114037860","state":["exists","up"]},{"osd":2,"uuid":"86378b92-f863-41ec-a3a5-f1dc78645250","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6800","nonce":2790378686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6801","nonce":2790378686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6803","nonce":2790378686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6802","nonce":2790378686}]},"public_addr":"172.21.15.154:6800/2790378686","cluster_addr":"172.21.15.154:6801/2790378686","heartbeat_back_addr":"172.21.15.154:6803/2790378686","heartbeat_front_addr":"172.21.15.154:6802/2790378686","state":["exists","up"]},{"osd":3,"uuid":"2fa93019-16da-44c6-ac77-0ef554e24e10","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6804","nonce":3602122863}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6805","nonce":3602122863}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6807","nonce":3602122863}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.154:6806","nonce":3602122863}]},"public_addr":"172.21.15.154:6804/3602122863","cluster_addr":"172.21.15.154:6805/3602122863","heartbeat_back_addr":"172.21.15.154:6807/3602122863","heartbeat_front_addr":"172.21.15.154:6806/3602122863","state":["exists","up"]},{"osd":4,"uuid":"08750d7a-1ca6-4d68-a62e-893e8d979921","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6812","nonce":1597208930}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6813","nonce":1597208930}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6815","nonce":1597208930}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6814","nonce":1597208930}]},"public_addr":"172.21.15.183:6812/1597208930","cluster_addr":"172.21.15.183:6813/1597208930","heartbeat_back_addr":"172.21.15.183:6815/1597208930","heartbeat_front_addr":"172.21.15.183:6814/1597208930","state":["exists","up"]},{"osd":5,"uuid":"770b03a7-7f15-4427-81ce-e512b68bbb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":22,"up_from":77,"up_thru":97,"down_at":24,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6800","nonce":40330014}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6801","nonce":40330014}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6803","nonce":40330014}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6802","nonce":40330014}]},"public_addr":"172.21.15.183:6800/40330014","cluster_addr":"172.21.15.183:6801/40330014","heartbeat_back_addr":"172.21.15.183:6803/40330014","heartbeat_front_addr":"172.21.15.183:6802/40330014","state":["exists","up"]},{"osd":6,"uuid":"ac82b5c6-7db8-4383-b0ac-0a203a4d84cd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":71,"up_from":81,"up_thru":15,"down_at":73,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6804","nonce":2007232062}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6805","nonce":2007232062}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6807","nonce":2007232062}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6806","nonce":2007232062}]},"public_addr":"172.21.15.183:6804/2007232062","cluster_addr":"172.21.15.183:6805/2007232062","heartbeat_back_addr":"172.21.15.183:6807/2007232062","heartbeat_front_addr":"172.21.15.183:6806/2007232062","state":["exists","up"]},{"osd":7,"uuid":"813f2f3d-68eb-487b-89a1-8b9a5a103485","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":13,"last_clean_end":66,"up_from":85,"up_thru":86,"down_at":67,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6808","nonce":2803215270}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6809","nonce":2803215270}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6811","nonce":2803215270}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.183:6810","nonce":2803215270}]},"public_addr":"172.21.15.183:6808/2803215270","cluster_addr":"172.21.15.183:6809/2803215270","heartbeat_back_addr":"172.21.15.183:6811/2803215270","heartbeat_front_addr":"172.21.15.183:6810/2803215270","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.817172+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.851798+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.765787+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.790359+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.824178+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.696786+0000","dead_epoch":24},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.761200+0000","dead_epoch":73},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T17:16:29.764552+0000","dead_epoch":67}],"pg_upmap":[{"pgid":"2.7","osds":[2,3]}],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T17:21:00.388 INFO:tasks.ceph:Scrubbing osd.0 2024-05-08T17:21:00.388 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:00.515 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:00.515 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:00.515 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:00.526 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-05-08T17:21:00.813 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 0 to deep-scrub 2024-05-08T17:21:00.826 INFO:tasks.ceph:Scrubbing osd.1 2024-05-08T17:21:00.827 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:00.952 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:00.953 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:00.953 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:00.963 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-05-08T17:21:01.252 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 1 to deep-scrub 2024-05-08T17:21:01.265 INFO:tasks.ceph:Scrubbing osd.2 2024-05-08T17:21:01.266 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:01.391 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:01.391 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:01.391 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:01.402 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-05-08T17:21:01.686 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 2 to deep-scrub 2024-05-08T17:21:01.700 INFO:tasks.ceph:Scrubbing osd.3 2024-05-08T17:21:01.700 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:01.823 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:01.824 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:01.824 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:01.834 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-05-08T17:21:02.121 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 3 to deep-scrub 2024-05-08T17:21:02.134 INFO:tasks.ceph:Scrubbing osd.4 2024-05-08T17:21:02.134 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:02.260 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:02.260 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:02.260 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:02.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2024-05-08T17:21:02.555 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 4 to deep-scrub 2024-05-08T17:21:02.567 INFO:tasks.ceph:Scrubbing osd.5 2024-05-08T17:21:02.568 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:02.693 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:02.693 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:02.693 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:02.704 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2024-05-08T17:21:02.990 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 5 to deep-scrub 2024-05-08T17:21:03.003 INFO:tasks.ceph:Scrubbing osd.6 2024-05-08T17:21:03.004 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:03.131 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:03.131 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:03.131 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:03.142 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2024-05-08T17:21:03.429 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 6 to deep-scrub 2024-05-08T17:21:03.442 INFO:tasks.ceph:Scrubbing osd.7 2024-05-08T17:21:03.443 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2024-05-08T17:21:03.568 INFO:teuthology.orchestra.run.smithi154.stdout:{ 2024-05-08T17:21:03.569 INFO:teuthology.orchestra.run.smithi154.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-05-08T17:21:03.569 INFO:teuthology.orchestra.run.smithi154.stdout:} 2024-05-08T17:21:03.579 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2024-05-08T17:21:03.868 INFO:teuthology.orchestra.run.smithi154.stderr:instructed osd(s) 7 to deep-scrub 2024-05-08T17:21:03.881 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:21:04.170 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:21:04.171 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:21:04.183 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":230,"stamp":"2024-05-08T17:21:02.752498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227688,"kb_used_data":9008,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747032,"statfs":{"total":773094113280,"available":772860960768,"internally_reserved":0,"allocated":9224192,"data_stored":4307440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.699353"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:02.670839+0000","last_change":"2024-05-08T17:21:02.670839+0000","last_active":"2024-05-08T17:21:02.670839+0000","last_peered":"2024-05-08T17:21:02.670839+0000","last_clean":"2024-05-08T17:21:02.670839+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:21:02.670839+0000","last_undegraded":"2024-05-08T17:21:02.670839+0000","last_fullsized":"2024-05-08T17:21:02.670839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:02.670716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:02.670716+0000","last_clean_scrub_stamp":"2024-05-08T17:21:02.670716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:30:01.111031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":186,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442459+0000","last_change":"2024-05-08T17:20:34.090566+0000","last_active":"2024-05-08T17:20:55.442459+0000","last_peered":"2024-05-08T17:20:55.442459+0000","last_clean":"2024-05-08T17:20:55.442459+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:20:55.442459+0000","last_undegraded":"2024-05-08T17:20:55.442459+0000","last_fullsized":"2024-05-08T17:20:55.442459+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:40:56.778107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":39,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153844+0000","last_change":"2024-05-08T17:20:39.956355+0000","last_active":"2024-05-08T17:20:55.153844+0000","last_peered":"2024-05-08T17:20:55.153844+0000","last_clean":"2024-05-08T17:20:55.153844+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:20:55.153844+0000","last_undegraded":"2024-05-08T17:20:55.153844+0000","last_fullsized":"2024-05-08T17:20:55.153844+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T18:00:33.709123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":189,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.442760+0000","last_change":"2024-05-08T17:18:42.932127+0000","last_active":"2024-05-08T17:20:55.442760+0000","last_peered":"2024-05-08T17:20:55.442760+0000","last_clean":"2024-05-08T17:20:55.442760+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:20:55.442760+0000","last_undegraded":"2024-05-08T17:20:55.442760+0000","last_fullsized":"2024-05-08T17:20:55.442760+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:29:01.953445+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":143,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743094+0000","last_change":"2024-05-08T17:20:51.435829+0000","last_active":"2024-05-08T17:20:55.743094+0000","last_peered":"2024-05-08T17:20:55.743094+0000","last_clean":"2024-05-08T17:20:55.743094+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:20:55.743094+0000","last_undegraded":"2024-05-08T17:20:55.743094+0000","last_fullsized":"2024-05-08T17:20:55.743094+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:47:48.244648+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:01.705266+0000","last_change":"2024-05-08T17:21:01.705302+0000","last_active":"2024-05-08T17:21:01.705266+0000","last_peered":"2024-05-08T17:21:01.705266+0000","last_clean":"2024-05-08T17:21:01.705266+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:21:01.705266+0000","last_undegraded":"2024-05-08T17:21:01.705266+0000","last_fullsized":"2024-05-08T17:21:01.705266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:01.705188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:01.705188+0000","last_clean_scrub_stamp":"2024-05-08T17:21:01.705188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:40.752697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":135,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153939+0000","last_change":"2024-05-08T17:20:39.233494+0000","last_active":"2024-05-08T17:20:55.153939+0000","last_peered":"2024-05-08T17:20:55.153939+0000","last_clean":"2024-05-08T17:20:55.153939+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:20:55.153939+0000","last_undegraded":"2024-05-08T17:20:55.153939+0000","last_fullsized":"2024-05-08T17:20:55.153939+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:56:58.553043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":130,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.743101+0000","last_change":"2024-05-08T17:20:51.435645+0000","last_active":"2024-05-08T17:20:55.743101+0000","last_peered":"2024-05-08T17:20:55.743101+0000","last_clean":"2024-05-08T17:20:55.743101+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:20:55.743101+0000","last_undegraded":"2024-05-08T17:20:55.743101+0000","last_fullsized":"2024-05-08T17:20:55.743101+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:33.795760+0000","last_clean_scrub_stamp":"2024-05-08T17:16:33.795760+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:15:11.404559+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"16'117","reported_seq":285,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:20:55.153794+0000","last_change":"2024-05-08T17:20:38.957787+0000","last_active":"2024-05-08T17:20:55.153794+0000","last_peered":"2024-05-08T17:20:55.153794+0000","last_clean":"2024-05-08T17:20:55.153794+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:20:55.153794+0000","last_undegraded":"2024-05-08T17:20:55.153794+0000","last_fullsized":"2024-05-08T17:20:55.153794+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:16:32.786322+0000","last_clean_scrub_stamp":"2024-05-08T17:16:32.786322+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:09:56.573083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542204,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28024,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343816,"statfs":{"total":96636764160,"available":96608067584,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542201,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542201,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28540,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343300,"statfs":{"total":96636764160,"available":96607539200,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220168,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29244,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342596,"statfs":{"total":96636764160,"available":96606818304,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350984,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481801,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28980,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342860,"statfs":{"total":96636764160,"available":96607088640,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574907,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574905,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-05-08T17:16:33.795760+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=33, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-05-08T17:16:32.786322+0000 time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=16, tm_sec=32, tm_wday=2, tm_yday=129, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=5, tm_mday=8, tm_hour=17, tm_min=20, tm_sec=59, tm_wday=2, tm_yday=129, tm_isdst=0) 2024-05-08T17:21:04.187 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-05-08T17:21:24.188 DEBUG:teuthology.orchestra.run.smithi154:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-05-08T17:21:24.486 INFO:teuthology.orchestra.run.smithi154.stdout: 2024-05-08T17:21:24.486 INFO:teuthology.orchestra.run.smithi154.stderr:dumped all 2024-05-08T17:21:24.498 INFO:teuthology.orchestra.run.smithi154.stdout:{"pg_ready":true,"pg_map":{"version":240,"stamp":"2024-05-08T17:21:22.756926+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":227644,"kb_used_data":9008,"kb_used_omap":20,"kb_used_meta":218539,"kb_avail":754747076,"statfs":{"total":773094113280,"available":772861005824,"internally_reserved":0,"allocated":9224192,"data_stored":4307440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":21177,"internal_metadata":223784263},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002841"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":132,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:02.670839+0000","last_change":"2024-05-08T17:21:02.670839+0000","last_active":"2024-05-08T17:21:02.670839+0000","last_peered":"2024-05-08T17:21:02.670839+0000","last_clean":"2024-05-08T17:21:02.670839+0000","last_became_active":"2024-05-08T17:18:50.418943+0000","last_became_peered":"2024-05-08T17:18:50.418943+0000","last_unstale":"2024-05-08T17:21:02.670839+0000","last_undegraded":"2024-05-08T17:21:02.670839+0000","last_fullsized":"2024-05-08T17:21:02.670839+0000","mapping_epoch":44,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":45,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:02.670716+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:02.670716+0000","last_clean_scrub_stamp":"2024-05-08T17:21:02.670716+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:30:01.111031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":195,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:02.353193+0000","last_change":"2024-05-08T17:21:02.353212+0000","last_active":"2024-05-08T17:21:02.353193+0000","last_peered":"2024-05-08T17:21:02.353193+0000","last_clean":"2024-05-08T17:21:02.353193+0000","last_became_active":"2024-05-08T17:20:34.090275+0000","last_became_peered":"2024-05-08T17:20:34.090275+0000","last_unstale":"2024-05-08T17:21:02.353193+0000","last_undegraded":"2024-05-08T17:21:02.353193+0000","last_fullsized":"2024-05-08T17:21:02.353193+0000","mapping_epoch":81,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":82,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:02.353131+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:02.353131+0000","last_clean_scrub_stamp":"2024-05-08T17:21:02.353131+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T04:46:29.638462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":48,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:06.632306+0000","last_change":"2024-05-08T17:21:06.632344+0000","last_active":"2024-05-08T17:21:06.632306+0000","last_peered":"2024-05-08T17:21:06.632306+0000","last_clean":"2024-05-08T17:21:06.632306+0000","last_became_active":"2024-05-08T17:20:39.956102+0000","last_became_peered":"2024-05-08T17:20:39.956102+0000","last_unstale":"2024-05-08T17:21:06.632306+0000","last_undegraded":"2024-05-08T17:21:06.632306+0000","last_fullsized":"2024-05-08T17:21:06.632306+0000","mapping_epoch":86,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":87,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:06.632214+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:06.632214+0000","last_clean_scrub_stamp":"2024-05-08T17:21:06.632214+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T20:41:37.677263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":198,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:01.376333+0000","last_change":"2024-05-08T17:21:01.376352+0000","last_active":"2024-05-08T17:21:01.376333+0000","last_peered":"2024-05-08T17:21:01.376333+0000","last_clean":"2024-05-08T17:21:01.376333+0000","last_became_active":"2024-05-08T17:18:42.931829+0000","last_became_peered":"2024-05-08T17:18:42.931829+0000","last_unstale":"2024-05-08T17:21:01.376333+0000","last_undegraded":"2024-05-08T17:21:01.376333+0000","last_fullsized":"2024-05-08T17:21:01.376333+0000","mapping_epoch":39,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":40,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:01.376267+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:01.376267+0000","last_clean_scrub_stamp":"2024-05-08T17:21:01.376267+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T03:48:30.543227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":152,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:04.892320+0000","last_change":"2024-05-08T17:21:04.892344+0000","last_active":"2024-05-08T17:21:04.892320+0000","last_peered":"2024-05-08T17:21:04.892320+0000","last_clean":"2024-05-08T17:21:04.892320+0000","last_became_active":"2024-05-08T17:20:51.435301+0000","last_became_peered":"2024-05-08T17:20:51.435301+0000","last_unstale":"2024-05-08T17:21:04.892320+0000","last_undegraded":"2024-05-08T17:21:04.892320+0000","last_fullsized":"2024-05-08T17:21:04.892320+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:04.892239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:04.892239+0000","last_clean_scrub_stamp":"2024-05-08T17:21:04.892239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:27:18.304551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":191,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:01.705266+0000","last_change":"2024-05-08T17:21:01.705302+0000","last_active":"2024-05-08T17:21:01.705266+0000","last_peered":"2024-05-08T17:21:01.705266+0000","last_clean":"2024-05-08T17:21:01.705266+0000","last_became_active":"2024-05-08T17:16:34.858716+0000","last_became_peered":"2024-05-08T17:16:34.858716+0000","last_unstale":"2024-05-08T17:21:01.705266+0000","last_undegraded":"2024-05-08T17:21:01.705266+0000","last_fullsized":"2024-05-08T17:21:01.705266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:01.705188+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:01.705188+0000","last_clean_scrub_stamp":"2024-05-08T17:21:01.705188+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T00:59:40.752697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":144,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:05.600472+0000","last_change":"2024-05-08T17:21:05.600493+0000","last_active":"2024-05-08T17:21:05.600472+0000","last_peered":"2024-05-08T17:21:05.600472+0000","last_clean":"2024-05-08T17:21:05.600472+0000","last_became_active":"2024-05-08T17:20:39.233226+0000","last_became_peered":"2024-05-08T17:20:39.233226+0000","last_unstale":"2024-05-08T17:21:05.600472+0000","last_undegraded":"2024-05-08T17:21:05.600472+0000","last_fullsized":"2024-05-08T17:21:05.600472+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:05.600373+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:05.600373+0000","last_clean_scrub_stamp":"2024-05-08T17:21:05.600373+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-10T01:32:28.735680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":139,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:03.854227+0000","last_change":"2024-05-08T17:21:03.854273+0000","last_active":"2024-05-08T17:21:03.854227+0000","last_peered":"2024-05-08T17:21:03.854227+0000","last_clean":"2024-05-08T17:21:03.854227+0000","last_became_active":"2024-05-08T17:20:51.435379+0000","last_became_peered":"2024-05-08T17:20:51.435379+0000","last_unstale":"2024-05-08T17:21:03.854227+0000","last_undegraded":"2024-05-08T17:21:03.854227+0000","last_fullsized":"2024-05-08T17:21:03.854227+0000","mapping_epoch":97,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":98,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T17:21:03.854132+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T17:21:03.854132+0000","last_clean_scrub_stamp":"2024-05-08T17:21:03.854132+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T21:41:24.404617+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"102'119","reported_seq":296,"reported_epoch":102,"state":"active+clean","last_fresh":"2024-05-08T17:21:04.643047+0000","last_change":"2024-05-08T17:21:04.643118+0000","last_active":"2024-05-08T17:21:04.643047+0000","last_peered":"2024-05-08T17:21:04.643047+0000","last_clean":"2024-05-08T17:21:04.643047+0000","last_became_active":"2024-05-08T17:20:38.957423+0000","last_became_peered":"2024-05-08T17:20:38.957423+0000","last_unstale":"2024-05-08T17:21:04.643047+0000","last_undegraded":"2024-05-08T17:21:04.643047+0000","last_fullsized":"2024-05-08T17:21:04.643047+0000","mapping_epoch":85,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":86,"parent":"0.0","parent_split_bits":0,"last_scrub":"102'119","last_scrub_stamp":"2024-05-08T17:21:04.642923+0000","last_deep_scrub":"102'119","last_deep_scrub_stamp":"2024-05-08T17:21:04.642923+0000","last_clean_scrub_stamp":"2024-05-08T17:21:04.642923+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T22:31:46.424216+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":528928,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1597440,"data_stored":1586784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":14,"seq":60129542208,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1592,"internal_metadata":27654600},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542205,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":14,"seq":60129542205,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28532,"kb_used_data":1516,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343308,"statfs":{"total":96636764160,"available":96607547392,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":85,"seq":365072220172,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29236,"kb_used_data":1516,"kb_used_omap":4,"kb_used_meta":27707,"kb_avail":94342604,"statfs":{"total":96636764160,"available":96606826496,"internally_reserved":0,"allocated":1552384,"data_stored":935126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28372685},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":81,"seq":347892350988,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27835,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4403,"internal_metadata":28503757},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":77,"seq":330712481805,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28972,"kb_used_data":996,"kb_used_omap":4,"kb_used_meta":27963,"kb_avail":94342868,"statfs":{"total":96636764160,"available":96607096832,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":4399,"internal_metadata":28634833},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":13,"seq":55834574911,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":28020,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343820,"statfs":{"total":96636764160,"available":96608071680,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":13,"seq":55834574909,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28012,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343828,"statfs":{"total":96636764160,"available":96608079872,"internally_reserved":0,"allocated":1019904,"data_stored":406198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":532480,"data_stored":528928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T17:21:24.500 DEBUG:teuthology.orchestra.run.smithi154:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-05-08T17:21:24.806 INFO:teuthology.misc:Shutting down mds daemons... 2024-05-08T17:21:24.806 INFO:teuthology.misc:Shutting down osd daemons... 2024-05-08T17:21:24.806 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-05-08T17:21:24.807 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:24.839 INFO:tasks.ceph.osd.0:Stopped 2024-05-08T17:21:24.840 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-05-08T17:21:24.840 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:24.918 INFO:tasks.ceph.osd.1:Stopped 2024-05-08T17:21:24.918 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-05-08T17:21:24.918 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:24.987 INFO:tasks.ceph.osd.2:Stopped 2024-05-08T17:21:24.987 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-05-08T17:21:24.987 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.060 INFO:tasks.ceph.osd.3:Stopped 2024-05-08T17:21:25.060 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2024-05-08T17:21:25.060 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.096 INFO:tasks.ceph.osd.4:Stopped 2024-05-08T17:21:25.097 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2024-05-08T17:21:25.097 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.175 INFO:tasks.ceph.osd.5:Stopped 2024-05-08T17:21:25.175 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2024-05-08T17:21:25.175 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.243 INFO:tasks.ceph.osd.6:Stopped 2024-05-08T17:21:25.243 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2024-05-08T17:21:25.243 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.321 INFO:tasks.ceph.osd.7:Stopped 2024-05-08T17:21:25.321 INFO:teuthology.misc:Shutting down mgr daemons... 2024-05-08T17:21:25.321 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2024-05-08T17:21:25.321 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.347 INFO:tasks.ceph.mgr.y:Stopped 2024-05-08T17:21:25.347 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-05-08T17:21:25.348 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.389 INFO:tasks.ceph.mgr.x:Stopped 2024-05-08T17:21:25.389 INFO:teuthology.misc:Shutting down mon daemons... 2024-05-08T17:21:25.390 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-05-08T17:21:25.390 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.401 INFO:tasks.ceph.mon.a:Stopped 2024-05-08T17:21:25.401 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2024-05-08T17:21:25.401 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.453 INFO:tasks.ceph.mon.c:Stopped 2024-05-08T17:21:25.454 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2024-05-08T17:21:25.454 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T17:21:25.466 INFO:tasks.ceph.mon.b:Stopped 2024-05-08T17:21:25.466 INFO:tasks.ceph:Checking cluster log for badness... 2024-05-08T17:21:25.466 DEBUG:teuthology.orchestra.run.smithi154:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'but it is still running' | egrep -v 'objects unfound and apparently lost' | egrep -v 'overall HEALTH_' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(OSD_' | egrep -v '\(PG_' | egrep -v '\(POOL_' | egrep -v '\(CACHE_POOL_' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(OBJECT_' | egrep -v '\(SLOW_OPS\)' | egrep -v '\(REQUEST_SLOW\)' | egrep -v '\(TOO_FEW_PGS\)' | egrep -v 'slow request' | egrep -v 'timeout on replica' | egrep -v 'late reservation from' | egrep -v MON_DOWN | egrep -v OSDMAP_FLAGS | egrep -v OSD_DOWN | egrep -v PG_DEGRADED | egrep -v PG_AVAILABILITY | egrep -v POOL_APP_NOT_ENABLED | egrep -v '\(OSD_SLOW_PING_TIME' | egrep -v 'overall HEALTH_' | egrep -v '\(MON_DOWN\)' | egrep -v '\(OSDMAP_FLAGS\)' | egrep -v '\(SMALLER_PGP_NUM\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v '\(SLOW OPS\)' | egrep -v 'slow request' | head -n 1 2024-05-08T17:21:25.526 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:21:25.526 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-05-08T17:21:25.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:21:25.717 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-05-08T17:21:25.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:21:25.799 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-05-08T17:21:25.857 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:21:25.857 DEBUG:teuthology.orchestra.run.smithi154:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-05-08T17:21:25.916 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:21:25.916 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2024-05-08T17:21:26.031 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:21:26.031 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2024-05-08T17:21:26.102 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:21:26.102 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2024-05-08T17:21:26.153 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:21:26.153 DEBUG:teuthology.orchestra.run.smithi183:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2024-05-08T17:21:26.201 INFO:tasks.ceph:Archiving mon data... 2024-05-08T17:21:26.203 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/data/mon.a.tgz 2024-05-08T17:21:26.203 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-05-08T17:21:26.219 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.fhjVA2Gxrl 2024-05-08T17:21:26.219 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.fhjVA2Gxrl 2024-05-08T17:21:26.355 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.fhjVA2Gxrl 2024-05-08T17:21:26.438 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.fhjVA2Gxrl is 378KB 2024-05-08T17:21:26.534 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.fhjVA2Gxrl 2024-05-08T17:21:26.549 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/data/mon.c.tgz 2024-05-08T17:21:26.549 DEBUG:teuthology.orchestra.run.smithi154:> mktemp 2024-05-08T17:21:26.603 INFO:teuthology.orchestra.run.smithi154.stdout:/tmp/tmp.CAEG1LRnMj 2024-05-08T17:21:26.604 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.CAEG1LRnMj 2024-05-08T17:21:26.730 DEBUG:teuthology.orchestra.run.smithi154:> sudo chmod 0666 /tmp/tmp.CAEG1LRnMj 2024-05-08T17:21:26.813 DEBUG:teuthology.orchestra.remote:smithi154:/tmp/tmp.CAEG1LRnMj is 378KB 2024-05-08T17:21:26.908 DEBUG:teuthology.orchestra.run.smithi154:> rm -fr /tmp/tmp.CAEG1LRnMj 2024-05-08T17:21:26.923 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/data/mon.b.tgz 2024-05-08T17:21:26.924 DEBUG:teuthology.orchestra.run.smithi183:> mktemp 2024-05-08T17:21:26.942 INFO:teuthology.orchestra.run.smithi183.stdout:/tmp/tmp.a5OxEZYILP 2024-05-08T17:21:26.943 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.a5OxEZYILP 2024-05-08T17:21:27.093 DEBUG:teuthology.orchestra.run.smithi183:> sudo chmod 0666 /tmp/tmp.a5OxEZYILP 2024-05-08T17:21:27.178 DEBUG:teuthology.orchestra.remote:smithi183:/tmp/tmp.a5OxEZYILP is 378KB 2024-05-08T17:21:27.249 DEBUG:teuthology.orchestra.run.smithi183:> rm -fr /tmp/tmp.a5OxEZYILP 2024-05-08T17:21:27.264 INFO:tasks.ceph:Cleaning ceph cluster... 2024-05-08T17:21:27.264 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T17:21:27.266 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-05-08T17:21:27.345 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T17:21:27.384 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-05-08T17:21:27.385 INFO:tasks.ceph:Archiving crash dumps... 2024-05-08T17:21:27.386 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/lib/ceph/crash to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi154/crash 2024-05-08T17:21:27.387 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T17:21:27.412 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/lib/ceph/crash to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi183/crash 2024-05-08T17:21:27.413 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-05-08T17:21:27.448 INFO:tasks.ceph:Compressing logs... 2024-05-08T17:21:27.449 DEBUG:teuthology.orchestra.run.smithi154:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T17:21:27.457 DEBUG:teuthology.orchestra.run.smithi183:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T17:21:27.478 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58418.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.56592.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.56592.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.55496.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.55496.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.58168.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32333.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph.audit.log: /var/log/ceph/objectstore_tool.58168.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.7.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32333.log: gzip -5 0.0% --verbose -- /var/log/ceph/ceph-osd.5.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32333.log.gz 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32162.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32202.log 2024-05-08T17:21:27.528 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32162.log: 0.0% 89.0% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.4.log 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32162.log.gz 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32202.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32202.log.gz 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.59682.log 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.53781.log 2024-05-08T17:21:27.529 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.59682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32413.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.53781.log: 93.5% -- replaced with /var/log/ceph/objectstore_tool.56592.log.gz 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/objectstore_tool.55496.log.gz 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32413.log: 0.0%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.41318.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr: -- replaced with /var/log/ceph/ceph-client.admin.32413.log.gz 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.6.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/objectstore_tool.41318.log: 93.4% -- replaced with /var/log/ceph/objectstore_tool.58168.log.gz 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.54618.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-osd.6.log: 93.1% -- replaced with /var/log/ceph/objectstore_tool.53781.log.gz 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32122.log 2024-05-08T17:21:27.539 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/objectstore_tool.59682.log.gz 2024-05-08T17:21:27.545 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.54618.log: 93.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-05-08T17:21:27.545 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.58925.log 2024-05-08T17:21:27.545 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32122.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32122.log.gzgzip -5 --verbose -- /var/log/ceph/objectstore_tool.57371.log 2024-05-08T17:21:27.545 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-05-08T17:21:27.551 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.58925.log: 93.1% -- replaced with /var/log/ceph/objectstore_tool.41318.log.gz 2024-05-08T17:21:27.551 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.42235.log 2024-05-08T17:21:27.551 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.57371.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52107.log 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38819.log 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58418.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65289.log 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58418.log.gz 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38819.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69261.log 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38819.log.gz 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65289.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68472.log 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65289.log.gz 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69261.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58953.log 0.0% 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69261.log.gz 2024-05-08T17:21:27.555 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68472.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48627.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68472.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58953.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66300.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58953.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47338.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48627.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5/var/log/ceph/ceph-client.admin.66300.log: --verbose -- /var/log/ceph/ceph-client.admin.55381.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66300.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose --/var/log/ceph/ceph-client.admin.47338.log: /var/log/ceph/ceph-client.admin.64728.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47338.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55381.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62097.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55381.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56163.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64728.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64728.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67329.log 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62097.log.gz 2024-05-08T17:21:27.556 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56163.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48657.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56163.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44319.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67329.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48657.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41209.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48657.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43417.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44319.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.41209.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49496.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41209.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.43417.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68810.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43417.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47676.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49496.log.gz 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55341.log 2024-05-08T17:21:27.557 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68810.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67875.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47676.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55341.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64447.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55341.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63817.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67875.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67875.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64447.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56342.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64447.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63817.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68715.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63817.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56342.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38811.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56342.log.gz 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47393.log 2024-05-08T17:21:27.558 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68715.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38811.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68854.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38811.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51996.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47393.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47393.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68854.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52121.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68854.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.51996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64324.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51996.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55702.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52121.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52121.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69765.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64324.log.gz 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55702.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55039.log 2024-05-08T17:21:27.559 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55702.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69765.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66617.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69765.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61823.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55039.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.66617.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40057.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66617.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61823.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.26350.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61823.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50200.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40057.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.tmp-client.admin.26350.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.26350.log.gz 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50200.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54553.log 2024-05-08T17:21:27.560 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50200.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63439.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53395.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54553.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67833.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63439.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40243.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53395.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49268.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67833.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40243.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40243.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47830.log 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr: 92.6% -- replaced with /var/log/ceph/ceph.log.gz 2024-05-08T17:21:27.561 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49268.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69933.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49268.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47830.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39671.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47830.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69933.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69933.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39671.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52684.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39671.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63319.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52684.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64496.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52684.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:gzip/var/log/ceph/ceph-client.admin.63319.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.39567.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63319.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39399.log 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64496.log.gz 2024-05-08T17:21:27.562 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39567.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57004.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39567.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39399.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55803.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39399.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57004.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68855.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57004.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55803.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55803.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47020.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59670.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68855.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47020.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44057.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47020.log.gz 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59670.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39021.log 2024-05-08T17:21:27.563 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59670.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44057.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38123.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44057.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39021.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62353.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39021.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57403.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38123.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62353.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62353.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38477.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57403.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40409.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57403.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38477.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63641.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38477.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40409.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65822.log 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40409.log.gz 2024-05-08T17:21:27.564 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67287.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63641.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51600.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65822.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67287.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67287.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.51600.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61614.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51600.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43527.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67198.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61614.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.43527.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43527.log.gz 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68479.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67198.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57799.log 2024-05-08T17:21:27.565 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67198.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68479.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68479.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39138.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57799.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49722.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57799.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39138.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39138.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40451.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49722.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39813.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr: 93.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49722.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40451.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60974.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40451.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39813.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56805.log 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39813.log.gz 2024-05-08T17:21:27.566 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63898.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60974.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56805.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56723.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56805.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63898.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55498.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63898.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42311.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56723.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55498.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53627.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42311.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44658.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42311.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53627.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54650.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53627.log.gz 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48327.log 2024-05-08T17:21:27.567 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44658.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54650.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54650.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52863.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48327.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69975.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48327.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52863.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38480.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52863.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69975.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39812.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69975.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38480.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39141.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38480.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51258.log 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39812.log.gz 2024-05-08T17:21:27.568 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39141.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55859.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39141.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.51258.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51258.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70059.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55859.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55859.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69219.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.70059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68183.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70059.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69219.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53928.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68183.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67665.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68183.log.gz 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53928.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64966.log 2024-05-08T17:21:27.569 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53928.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67665.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44278.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67665.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64966.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64966.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48499.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44278.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68182.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44278.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48499.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48499.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64848.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68518.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68182.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64848.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64848.log.gz 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56531.log 2024-05-08T17:21:27.570 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68518.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39357.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68518.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56531.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.56531.log.gz -5 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.50734.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54112.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39357.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38487.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: 27.2% -- replaced with /var/log/ceph/ceph-client.admin.50734.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54112.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55496.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54112.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38487.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38487.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59115.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55496.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54957.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55496.log.gz 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59115.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59115.log.gz -5 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67240.log 2024-05-08T17:21:27.571 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54957.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62354.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54957.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67240.log: gzip -5 --verbose 0.0% -- -- replaced with /var/log/ceph/ceph-client.admin.67240.log.gz /var/log/ceph/ceph-client.admin.38489.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68462.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62354.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38489.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38489.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67156.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68462.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67581.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68462.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67156.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67156.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50336.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69723.log 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67581.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67581.log.gz 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69471.log 0.0% 2024-05-08T17:21:27.572 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50336.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69723.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56845.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69723.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69471.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 0.0% 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69471.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56845.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66873.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56845.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip 0.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.62438.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.66873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69135.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66873.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62438.log: gzip -5 0.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.62438.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.38488.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69135.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69135.log.gz 2024-05-08T17:21:27.573 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38488.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38488.log.gzgzip 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.68181.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.y.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68181.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49452.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68181.log.gz 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40949.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49452.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47751.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49452.log.gz 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54835.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40949.log.gz 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47751.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63986.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47751.log.gz 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54835.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68476.log 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54835.log.gz 2024-05-08T17:21:27.574 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63986.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63986.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68476.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62157.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68476.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57740.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.62157.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46914.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62157.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69807.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57740.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.46914.log: -- /var/log/ceph/ceph-client.admin.38485.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46914.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69807.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69807.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.61774.log 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38485.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38485.log.gz 2024-05-08T17:21:27.575 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40508.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61774.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60693.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40508.log: gzip 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40508.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.61492.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60693.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60693.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39525.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61492.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61492.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.60890.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39525.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39525.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40888.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60890.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60890.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.59161.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40888.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40888.log.gz 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40305.log 2024-05-08T17:21:27.576 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59161.log: gzip -5 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59161.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr: --verbose -- /var/log/ceph/ceph-client.admin.67749.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40305.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40305.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58213.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67749.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.c.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58213.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38307.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65781.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39181.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38307.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65781.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65781.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53908.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62615.log 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39181.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39181.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53908.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53908.log.gz 2024-05-08T17:21:27.577 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38265.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62615.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62615.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38022.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39609.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38265.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38265.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38022.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38022.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61853.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39609.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39609.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56260.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39139.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61853.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56260.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56260.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49212.log 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39139.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39139.log.gz 2024-05-08T17:21:27.578 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68812.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54897.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49212.log: 29.5% -- replaced with /var/log/ceph/ceph-client.admin.49212.log.gz 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68812.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68812.log.gz 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54897.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54897.log.gz 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67371.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69093.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-mon.a.log: /var/log/ceph/ceph-client.admin.67371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67371.log.gz 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69303.log 2024-05-08T17:21:27.579 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69093.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69093.log.gz 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.42235.log: 93.8%gzip -5 --verbose -- /var/log/ceph/objectstore_tool.52984.log 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.52107.log: -- replaced with /var/log/ceph/objectstore_tool.54618.log.gz 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr: 93.6% -- replaced with /var/log/ceph/objectstore_tool.58925.log.gz 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr: 93.4% -- replaced with /var/log/ceph/objectstore_tool.57371.log.gz 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/objectstore_tool.40541.log 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.52984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32287.log 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr: 93.1% -- replaced with /var/log/ceph/objectstore_tool.42235.log.gz 2024-05-08T17:21:27.589 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.40541.log: 93.2% -- replaced with /var/log/ceph/objectstore_tool.52107.log.gz 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32373.log 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32287.log: gzip -5 --verbose -- /var/log/ceph/objectstore_tool.39604.log 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32287.log.gz 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-client.admin.32373.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32373.log.gz 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.b.log 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/objectstore_tool.39604.log: 93.2% -- replaced with /var/log/ceph/objectstore_tool.40541.log.gz 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32242.log 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr:/var/log/ceph/ceph-mon.b.log: /var/log/ceph/ceph-client.admin.32242.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32242.log.gz 2024-05-08T17:21:27.590 INFO:teuthology.orchestra.run.smithi183.stderr: 93.0% -- replaced with /var/log/ceph/objectstore_tool.52984.log.gz 2024-05-08T17:21:27.604 INFO:teuthology.orchestra.run.smithi183.stderr: 93.9% -- replaced with /var/log/ceph/objectstore_tool.39604.log.gz 2024-05-08T17:21:27.680 INFO:teuthology.orchestra.run.smithi183.stderr: 90.9% -- replaced with /var/log/ceph/ceph-osd.4.log.gz 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52313.log 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69303.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69303.log.gz 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69681.log 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52313.log.gz 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67539.log 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69681.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69681.log.gz 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43135.log 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67539.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67539.log.gz 2024-05-08T17:21:27.684 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69513.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.43135.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43135.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52400.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69513.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69513.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56530.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52400.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52400.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66363.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56530.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56530.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62498.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.66363.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66363.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58593.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62498.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62498.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67455.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58593.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58593.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61955.log 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67455.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67455.log.gz 2024-05-08T17:21:27.685 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67707.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61955.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61955.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53264.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67707.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67707.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39441.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53264.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53264.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54772.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39441.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39441.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53324.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54772.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54772.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42132.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53324.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53324.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65686.log 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42132.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42132.log.gz 2024-05-08T17:21:27.686 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32566.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65686.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65686.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39972.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.32566.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32566.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50476.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39972.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67623.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50476.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50476.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64325.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67623.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67623.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50915.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64325.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64325.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69597.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50915.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50915.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68175.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69597.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69597.log.gz 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69387.log 2024-05-08T17:21:27.687 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68175.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68175.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63818.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69387.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69387.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59313.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63818.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63818.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58850.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59313.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59313.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38080.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58850.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58850.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57226.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38080.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38080.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52801.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57226.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57226.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69051.log 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52801.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52801.log.gz 2024-05-08T17:21:27.688 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52843.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69051.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69051.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39483.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52843.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52843.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52603.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39483.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67917.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52603.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52603.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61333.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67917.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69345.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61333.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61333.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62816.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69345.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69345.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49553.log 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62816.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62816.log.gz 2024-05-08T17:21:27.689 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68815.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49553.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49553.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69555.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68815.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44909.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69555.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69555.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55258.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44909.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44909.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68179.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55258.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55258.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61034.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68179.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68179.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54917.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61034.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61034.log.gz 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.51097.log 2024-05-08T17:21:27.690 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54917.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54917.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62817.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.51097.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.51097.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69849.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62817.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62817.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69849.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69849.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69429.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr: 91.4% -- replaced with /var/log/ceph/ceph-mgr.y.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39041.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64586.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69429.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40846.log 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69429.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr:gzip/var/log/ceph/ceph-client.admin.64586.log: -5 --verbose -- 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64586.log.gz 2024-05-08T17:21:27.691 INFO:teuthology.orchestra.run.smithi154.stderr: /var/log/ceph/ceph-client.admin.38824.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54371.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40846.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40846.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38824.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38824.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62899.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54371.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64103.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62899.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66025.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.62899.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40367.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64103.log: /var/log/ceph/ceph-client.admin.66025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66025.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64103.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65329.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40367.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40367.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67497.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38685.log 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65329.log: /var/log/ceph/ceph-client.admin.67497.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65329.log.gz 2024-05-08T17:21:27.692 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67497.log.gz 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60633.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38685.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38685.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59972.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54391.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60633.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60633.log.gz 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59972.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59972.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70017.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48232.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54391.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54391.log.gz 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.70017.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70017.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61372.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48407.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48232.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48232.log.gz 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61372.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61372.log.gz 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67959.log 2024-05-08T17:21:27.693 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50252.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48407.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48407.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67959.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67959.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59395.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39810.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50252.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50252.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59395.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59395.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68174.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42490.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39810.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39810.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68174.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68174.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32161.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54213.log 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42490.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42490.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.32161.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32161.log.gz 2024-05-08T17:21:27.694 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59650.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68811.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.59650.log: /var/log/ceph/ceph-client.admin.54213.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54213.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59650.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39140.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.68811.log: -- /var/log/ceph/ceph-client.admin.63942.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68811.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68809.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63942.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39140.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63942.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67791.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68809.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42853.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68809.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67791.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.67791.log.gz 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57820.log 2024-05-08T17:21:27.695 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40729.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42853.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42853.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57820.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57820.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61171.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40729.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38823.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65108.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61171.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61171.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38823.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38823.log.gzgzip 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.49792.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65108.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65108.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63098.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49792.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48130.log 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.49792.log.gz 2024-05-08T17:21:27.696 INFO:teuthology.orchestra.run.smithi154.stderr:gzip/var/log/ceph/ceph-client.admin.63098.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.64748.log 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63098.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57281.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.48130.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.48130.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.64748.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64748.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58274.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57281.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57281.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46956.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58274.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58274.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.57564.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42612.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.57564.log: /var/log/ceph/ceph-client.admin.46956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.57564.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.46956.log.gz 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69177.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42612.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42612.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40141.log 2024-05-08T17:21:27.697 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68180.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.40141.log: /var/log/ceph/ceph-client.admin.69177.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.40141.log.gz 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.69177.log.gz 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.62956.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68180.log: 0.0%gzip -- replaced with /var/log/ceph/ceph-client.admin.68180.log.gz -5 --verbose -- /var/log/ceph/ceph-client.admin.60531.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58714.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.62956.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.62956.log.gz 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60531.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38815.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.60531.log.gz 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53809.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58714.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58714.log.gz 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38815.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38815.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69891.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47297.log 2024-05-08T17:21:27.698 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53809.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.69891.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69891.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38825.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.55999.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.47297.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.47297.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38825.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38825.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63561.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.60886.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.55999.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.55999.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63561.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49329.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.63561.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53729.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.60886.log: /var/log/ceph/ceph-client.admin.49329.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49329.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.60886.log.gz 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50854.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53729.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53729.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53122.log 2024-05-08T17:21:27.699 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53025.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53122.log: /var/log/ceph/ceph-client.admin.50854.log: 0.0% 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53122.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.50854.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41892.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53025.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53025.log.gzgzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42172.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.49408.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.41892.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41892.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42172.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56443.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.42172.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56865.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.49408.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.49408.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56443.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68475.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56443.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56865.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.32463.log 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56865.log.gz 2024-05-08T17:21:27.700 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68475.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52283.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68475.log.gz/var/log/ceph/ceph-client.admin.32463.log: 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.32463.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.61435.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38219.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52283.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52283.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.61435.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.61435.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41491.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66637.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.41491.log: /var/log/ceph/ceph-client.admin.38219.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38219.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.41491.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.65371.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.66178.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.66637.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66637.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.65371.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.65371.log.gz 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.70099.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.50619.log 2024-05-08T17:21:27.701 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.70099.log: /var/log/ceph/ceph-client.admin.66178.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.66178.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.70099.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.42673.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53394.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.50619.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.50619.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42673.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.42673.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68001.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.64027.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68001.log: /var/log/ceph/ceph-client.admin.53394.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53394.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68001.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38473.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5/var/log/ceph/ceph-client.admin.64027.log: --verbose -- /var/log/ceph/ceph-client.admin.54752.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.64027.log.gz 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38483.log 2024-05-08T17:21:27.702 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38473.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38473.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54752.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54752.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58774.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44515.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38483.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38483.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.58774.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58774.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43938.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.56001.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.43938.log: /var/log/ceph/ceph-client.admin.44515.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.43938.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44515.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.52399.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68471.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.56001.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.56001.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.52399.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.52399.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63318.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.63320.log 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63318.log: /var/log/ceph/ceph-client.admin.68471.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63318.log.gz 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68471.log.gz 2024-05-08T17:21:27.703 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.54289.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38822.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.63320.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.63320.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.54289.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.54289.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39809.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.59035.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.38822.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.38822.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39809.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39809.log.gzgzip 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.42413.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44122.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.42413.log: /var/log/ceph/ceph-client.admin.59035.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.59035.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr: 28.5% -- replaced with /var/log/ceph/ceph-client.admin.42413.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.58039.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68176.log 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.44122.log: /var/log/ceph/ceph-client.admin.58039.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.44122.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.58039.log.gz 2024-05-08T17:21:27.704 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39183.log 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68176.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.69639.log 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.68176.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.67413.log 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.39183.log: /var/log/ceph/ceph-client.admin.69639.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.69639.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.39183.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.53749.log 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.67413.log: 0.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.68519.log 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr: -- replaced with /var/log/ceph/ceph-client.admin.67413.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.53749.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.53749.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr:/var/log/ceph/ceph-client.admin.68519.log: 0.0% -- replaced with /var/log/ceph/ceph-client.admin.68519.log.gz 2024-05-08T17:21:27.705 INFO:teuthology.orchestra.run.smithi154.stderr: 91.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-05-08T17:21:27.706 INFO:teuthology.orchestra.run.smithi154.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.c.log.gz 2024-05-08T17:21:27.720 INFO:teuthology.orchestra.run.smithi183.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.b.log.gz 2024-05-08T17:21:27.731 INFO:teuthology.orchestra.run.smithi183.stderr: 91.1% -- replaced with /var/log/ceph/ceph-osd.6.log.gz 2024-05-08T17:21:27.739 INFO:teuthology.orchestra.run.smithi183.stderr: 91.5% -- replaced with /var/log/ceph/ceph-osd.7.log.gz 2024-05-08T17:21:27.821 INFO:teuthology.orchestra.run.smithi154.stderr: 90.7% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-05-08T17:21:28.078 INFO:teuthology.orchestra.run.smithi183.stderr: 94.1% -- replaced with /var/log/ceph/ceph-osd.5.log.gz 2024-05-08T17:21:28.080 INFO:teuthology.orchestra.run.smithi183.stderr: 2024-05-08T17:21:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:real 0m0.575s 2024-05-08T17:21:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:user 0m1.487s 2024-05-08T17:21:28.080 INFO:teuthology.orchestra.run.smithi183.stderr:sys 0m0.075s 2024-05-08T17:21:28.122 INFO:teuthology.orchestra.run.smithi154.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-05-08T17:21:28.158 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-05-08T17:21:28.176 INFO:teuthology.orchestra.run.smithi154.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-05-08T17:21:28.178 INFO:teuthology.orchestra.run.smithi154.stderr: 2024-05-08T17:21:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:real 0m0.709s 2024-05-08T17:21:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:user 0m2.665s 2024-05-08T17:21:28.178 INFO:teuthology.orchestra.run.smithi154.stderr:sys 0m0.279s 2024-05-08T17:21:28.178 INFO:tasks.ceph:Archiving logs... 2024-05-08T17:21:28.178 DEBUG:teuthology.misc:Transferring archived files from smithi154:/var/log/ceph to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi154/log 2024-05-08T17:21:28.179 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T17:21:28.832 DEBUG:teuthology.misc:Transferring archived files from smithi183:/var/log/ceph to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi183/log 2024-05-08T17:21:28.833 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T17:21:29.089 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T17:21:29.099 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T17:21:29.099 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T17:21:29.102 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T17:21:29.183 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T17:21:29.194 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T17:21:29.194 DEBUG:teuthology.orchestra.run.smithi154:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T17:21:29.197 DEBUG:teuthology.orchestra.run.smithi183:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T17:21:29.209 INFO:teuthology.orchestra.run.smithi154.stderr:bash: line 1: ntpq: command not found 2024-05-08T17:21:29.213 INFO:teuthology.orchestra.run.smithi183.stderr:bash: line 1: ntpq: command not found 2024-05-08T17:21:29.213 INFO:teuthology.orchestra.run.smithi154.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T17:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout:=============================================================================== 2024-05-08T17:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout:^* hv01.front.sepia.ceph.com 3 6 377 46 -2101us[-2102us] +/- 57ms 2024-05-08T17:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 48 +1935us[+1934us] +/- 69ms 2024-05-08T17:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 47 +708us[ +707us] +/- 73ms 2024-05-08T17:21:29.214 INFO:teuthology.orchestra.run.smithi154.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:=============================================================================== 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:^* hv01.front.sepia.ceph.com 3 6 377 47 -2072us[-2075us] +/- 57ms 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 45 +1956us[+1956us] +/- 69ms 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 46 +744us[ +744us] +/- 73ms 2024-05-08T17:21:29.217 INFO:teuthology.orchestra.run.smithi183.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T17:21:29.218 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T17:21:29.227 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T17:21:29.228 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T17:21:29.237 DEBUG:teuthology.orchestra.run.smithi154:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T17:21:29.339 DEBUG:teuthology.orchestra.run.smithi183:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T17:21:29.416 DEBUG:teuthology.orchestra.run.smithi154:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:21:29.445 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:21:29.446 DEBUG:teuthology.orchestra.run.smithi183:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T17:21:29.475 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:21:29.475 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T17:21:29.485 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T17:21:29.493 INFO:teuthology.task.internal:Duration was 748.208955 seconds 2024-05-08T17:21:29.494 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T17:21:29.503 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T17:21:29.503 DEBUG:teuthology.orchestra.run.smithi154:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T17:21:29.506 DEBUG:teuthology.orchestra.run.smithi183:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T17:21:29.542 INFO:teuthology.orchestra.run.smithi154.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:21:29.554 INFO:teuthology.orchestra.run.smithi183.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T17:21:29.778 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T17:21:29.779 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi154.front.sepia.ceph.com 2024-05-08T17:21:29.779 DEBUG:teuthology.orchestra.run.smithi154:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T17:21:29.808 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi183.front.sepia.ceph.com 2024-05-08T17:21:29.808 DEBUG:teuthology.orchestra.run.smithi183:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T17:21:29.839 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T17:21:29.839 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T17:21:29.850 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T17:21:29.905 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T17:21:29.905 DEBUG:teuthology.orchestra.run.smithi154:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T17:21:29.916 DEBUG:teuthology.orchestra.run.smithi183:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T17:21:30.062 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T17:21:30.072 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T17:21:30.072 DEBUG:teuthology.orchestra.run.smithi154:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T17:21:30.097 DEBUG:teuthology.orchestra.run.smithi183:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T17:21:30.122 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T17:21:30.132 DEBUG:teuthology.orchestra.run.smithi154:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T17:21:30.139 DEBUG:teuthology.orchestra.run.smithi183:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T17:21:30.159 INFO:teuthology.orchestra.run.smithi154.stdout:kernel.core_pattern = core 2024-05-08T17:21:30.185 INFO:teuthology.orchestra.run.smithi183.stdout:kernel.core_pattern = core 2024-05-08T17:21:30.205 DEBUG:teuthology.orchestra.run.smithi154:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T17:21:30.239 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:21:30.240 DEBUG:teuthology.orchestra.run.smithi183:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T17:21:30.259 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T17:21:30.260 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T17:21:30.269 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T17:21:30.270 DEBUG:teuthology.misc:Transferring archived files from smithi154:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi154 2024-05-08T17:21:30.270 DEBUG:teuthology.orchestra.run.smithi154:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T17:21:30.316 DEBUG:teuthology.misc:Transferring archived files from smithi183:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/lflores-2024-05-08_14:59:36-rados-wip-lflores-testing-2-2024-05-07-1606-squid-distro-default-smithi/7697758/remote/smithi183 2024-05-08T17:21:30.316 DEBUG:teuthology.orchestra.run.smithi183:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T17:21:30.350 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T17:21:30.350 DEBUG:teuthology.orchestra.run.smithi154:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T17:21:30.352 DEBUG:teuthology.orchestra.run.smithi183:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T17:21:30.399 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T17:21:30.410 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T17:21:30.410 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T17:21:30.420 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T17:21:30.420 DEBUG:teuthology.orchestra.run.smithi154:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T17:21:30.423 DEBUG:teuthology.orchestra.run.smithi183:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T17:21:30.435 INFO:teuthology.orchestra.run.smithi154.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 17:21 /home/ubuntu/cephtest 2024-05-08T17:21:30.454 INFO:teuthology.orchestra.run.smithi183.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 17:21 /home/ubuntu/cephtest 2024-05-08T17:21:30.455 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T17:21:30.465 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T17:21:30.486 INFO:teuthology.run:Summary data: description: rados/verify/{centos_latest ceph clusters/{fixed-2 openstack} d-thrash/default/{default thrashosds-health} mon_election/connectivity msgr-failures/few msgr/async-v2only objectstore/bluestore-hybrid rados tasks/mon_recovery validater/lockdep} duration: 748.2089550495148 flavor: default owner: scheduled_lflores@teuthology success: true 2024-05-08T17:21:30.487 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T17:21:30.570 INFO:teuthology.run:pass